random bits of computer info

Monday, September 14, 2015

Copy files to running OpenShift 3 pods

Needed to copy a newer mysql client for debugging purposes into an OpenShift v3 pod. /fortunately the mysql client binary is self contained and does not need special shared libs; worked with v5.6.26/

Here's how I managed to do so:

oc exec -i [pod name] -n [project] -- bash -c 'cat > [remote file]' < [local file]

e.g.
oc exec -i mysql-55-centos7-1-f42js -n 0wwwv -- bash -c 'cat > /tmp/mysql' < /tmp/mysql

What does it do?

Invokes `exec` command with `-i` option so that stdin is passed (otherwise stdin is ignored). The remote command cats everything into desired path. You need to `chmod 755 [file]` inside the pod if that file is to be executed.

To get into the pod in a shell try `oc rsh`.

Tuesday, July 21, 2015

HTTP Proxy vs overriding hostname

There's known trick with `curl` to connect a server via IP address and override the virtual host you connect to using the `Host:` header. Or using the `--resolve` option. This is mainly useful to try test server instances that don't have proper DNS records set.

curl -H "Host: abc.example.com" 8.7.6.5
curl --resolve abc.example.com:80:8.7.6.5 http://abc.example.com

This is quiet handy. And in unfortunate circumstances one can need that but behind a proxy. So I was banging my head to understand how to do it. Eventually I resorted to looking at rfc2068.

The news are not good (as much as probably very little people care) - there is no way to perform that hack over a standard HTTP Proxy. Here's and excerpt:

5.2 The Resource Identified by a Request

   HTTP/1.1 origin servers SHOULD be aware that the exact resource
   identified by an Internet request is determined by examining both the
   Request-URI and the Host header field.

   An origin server that does not allow resources to differ by the
   requested host MAY ignore the Host header field value. (But see
   section 19.5.1 for other requirements on Host support in HTTP/1.1.)

   An origin server that does differentiate resources based on the host
   requested (sometimes referred to as virtual hosts or vanity
   hostnames) MUST use the following rules for determining the requested
   resource on an HTTP/1.1 request:

     1. If Request-URI is an absoluteURI, the host is part of the
        Request-URI. Any Host header field value in the request MUST be
        ignored.

     2. If the Request-URI is not an absoluteURI, and the request
        includes a Host header field, the host is determined by the Host
        header field value.

     3. If the host as determined by rule 1 or 2 is not a valid host on
        the server, the response MUST be a 400 (Bad Request) error
        message.

   Recipients of an HTTP/1.0 request that lacks a Host header field MAY
   attempt to use heuristics (e.g., examination of the URI path for
   something unique to a particular host) in order to determine what
   exact resource is being requested.

There are two ways to ask a HTTP proxy to perform a request:

GET http://abc.ecample.com/ HTTP 1.0

GET / HTTP 1.0
Host: abc.ecample.com

In the first form (point #1 above), even if `Host` header is provided, the proxy server must ignore it and send to the target web server whatever host is provided in the absolute URL. That's why it's technically impossible to do this over a HTTP proxy.

This doesn't mean you can't do it with a socks server or if proxy allows CONNECT to the desired destination port.

With socks perhaps `curl` should do just fine (haven't tested). Using CONNECT to port 80 though would require manual interaction with the socket or a little programming as standard tools would not use CONNECT for simple non-encrypted HTTP connections.

Premium content:
Here's how to use proxy with auth via telnet:
http://unix.stackexchange.com/a/36632/14907

And here's a good example how to use the CONNECT method manually:
http://blog.vivekjishtu.com/2013/09/using-connect-method-on-http-proxy.html

How to get certificate chain by ruby

Looking on the Internet I couldn't find any examples how to get a hold on a remote server cerificate chain. But it turned out to be rather simple looking at ruby 2.2.2 api docs.

https://gist.github.com/akostadinov/fa54e7bc5a1858af5677

Thought might be useful to somebody to post in public..

Thursday, July 9, 2015

Apache HTTPd SSL/OCSP primer

I just stumbled over some old howto I used for testing OSCP. Here I'm pasting it to hopefully help somebody understand how it works. It assumes you have already up and running Apache HTTPd web server.

Create needed certificates

Download this useful script:
https://git.fedorahosted.org/cgit/pkinit-nss.git/plain/doc/openssl/make-certs.sh
Use the script to generate certifictes:

./make-certs.sh europa.sfo.corp.google.com test@example.com all ocsp:http://europa.sfo.corp.google.com/
./make-certs.sh america.sfo.corp.google.com test@example.com all ocsp:http://europa.sfo.corp.google.com/

Prepare and start responder using openSSL

Create index.txt file which contains information about certificates the responder is handling.
File is plain text file with fields separated by TAB.
Here is example of structure:

V       100320100000Z           593C5290F246444B        unknown DC=com, DC=example/mail=test@example.com, CN=europa.sfo.corp.google.com
V                       B9290C71D224ACB3        unknown DC=com, DC=example, CN=Test Certifying CA
R       131021200751Z   100324142709Z,superseded        593C5290F246444C        unknown DC=com, DC=example/mail=test@example.com, CN=america.sfo.corp.google.com

The columns are defined as:
#define DB_type         0 /* Status of the certificate */
#define DB_exp_date     1 /* Expiry date */
#define DB_rev_date     2 /* Revocation date */
#define DB_serial       3       /* Serial No., index - unique */
#define DB_file         4      
#define DB_name         5       /* DN, index - unique when active and not disabled */

Notes:
- DB_type could be only V | R, E is not working
- date format is YYMMDDHHMMSSZ
- DB_serial has to be in HEX with upper case letters
- use example provided above, just don't forget to change serial numbers

Start responder using:

openssl ocsp -index index.txt -port 8088 -rsigner certs/ca.pem -CA certs/ca.pem -text

Check if responder is working fine using:

openssl ocsp -issuer certs/ca.pem -CAfile certs/ca.pem -url http://localhost:8088 -cert certs/europa.sfo.corp.google.com.pem

should return GOOD certificate status

openssl ocsp -issuer certs/ca.pem -CAfile certs/ca.pem -url http://localhost:8088 -cert certs/america.sfo.corp.google.com.pem

should return REVOKED ... reason: superseded

Modify $EWS_HOME/httpd/conf.d/ssl.conf:

- add this to the end of file:

#OCSP
SSLVerifyClient on
SSLVerifyDepth 10
SSLOCSPEnable on
SSLOCSPDefaultResponder http://localhost:8088/
SSLOCSPOverrideResponder on

- httpd has to ask for client certificate and enable OCSP and set default responder

- uncomment SSLCACertificateFile /etc/pki/tls/cert.pem line in the SSL configuration file and make sure your new CA is added to the cert.pem file
or just point this to oscp/certs/ca.crt (generated in one of previous steps)

- copy some static content to $EWS_HOME/httpd/www/html/
e.g. echo "OCSPTestSucceed" > $EWS_HOME/httpd/www/html/ocsp.txt

- start apache

Now try it out:

run:

wget --output-document=/dev/null --no-check-certificat --certificate=client_cert/america.sfo.corp.google.com.pem --ca-certificate=client_cert/ca.pem  https://<your server>/ocsp.txt

result:

Resolving <your server>... 10.34.34.43
Connecting to <your server>|10.34.34.43|:443... connected.
OpenSSL: error:14094414:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate revoked
Unable to establish SSL connection.

run:

wget --output-document=/dev/null --no-check-certificat --certificate=client_cert/europa.sfo.corp.google.com.pem --ca-certificate=client_cert/ca.pem  https://<your server>/ocsp.txt

result:

HTTP request sent, awaiting response... 200 OK

References

Attribution

Thanks to my colleague Rajesh for preparing initial info.

Wednesday, July 8, 2015

OpenShift v3 REST API usage

Time for a quick v3 api trial. The REST api is completely changed. First notable thing is authentication. Basic auth is no longer supported. Now the only supported auth types are oauth token and client SSL certificate.

Lets see how we can get an oauth token. First method is to go with a browser to https://your.openshift.master.server.example.com:8443/oauth/token/request

That will present you with a nice page explaining you how to use your newly acquired token. But we want to perform this in a more machine friendly manner so let's do with openshift-challenging-client via curl:

curl -u joe -kv -H "X-CSRF-Token: xxx" 'https://master.cluster.local:8443/oauth/authorize?client_id=openshift-challenging-client&response_type=token'

Keys here are:

use client_id=openshift-challenging-client, otherwise 400 Bad Request is returned
use response_type=token
set X-CSRF-Token header to some non-empty value, otherwise error is returned
actual token is returned in the Location header of the 302 response per the OAuth spec as access_token=VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU

See partial curl output:

< HTTP/1.1 302 Found
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Expires: Fri, 01 Jan 1990 00:00:00 GMT
< Location: https://master.cluster.local:8443/oauth/token/display#access_token=VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU&expires_in=86400&token_type=bearer
< Pragma: no-cache
< Set-Cookie: ssn=MTQzNjM3NzI4NXxDSkxSTl8yb0ZjUmZaSDZwNG51UjNDZEx1M29xRldQNGtGZTMwbnhfYlNRV2FuVmYxVHlKSWhWazVKWjR2RDc3X056ZVpqZXl6VWN4T0Nqc1dyX01raDhiUlNSdXFpdkhDalAwWDQzNWdyWExlTmNTUURjN3pQeW9HT1RpVmRtQ1JBPT18qi62Db0PolIHaMmAjtdKPejhCGRY-EUEruT6W_Du2bg=; Path=/; Expires=Wed, 08 Jul 2015 18:41:25 UTC; Max-Age=3600; HttpOnly; Secure
< Date: Wed, 08 Jul 2015 17:41:25 GMT
< Content-Length: 0
< Content-Type: text/plain; charset=utf-8

Now lets perform a real API call to remove the token we just obtained:

curl -vk -H "Authorization: Bearer VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU" https://master.cluster.local:8443/oapi/v1/oauthaccesstokens/VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU -X DELETE

Well done!

Notice how we pass the token - using header:

"Authorization: Bearer VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU"

You might have noticed API endpoint is /oapi/v1/something, where "v1" is the version of the API. How do we get server supported API versions? 2 ways so far:

$ curl -k https://master.cluster.local:8443/
{
"paths": [
    "/api",
    "/api/v1beta3",
    "/api/v1",
    "/controllers",
    "/healthz",
    "/healthz/ping",
    "/logs/",
    "/metrics",
    "/ready",
    "/osapi",
    "/osapi/v1beta3",
    "/oapi",
    "/oapi/v1",
    "/swaggerapi/"
]

Looking specifically at kubernetes API endpoint:

$ curl -vk https://master.cluster.local:8443/api
{
"versions": [
"v1beta3",
"v1"
]

Well forgot to mention. OpenShift v3 does give you access to the plain kubernetes REST API as well to the OpenShift REST API, because OpenShift is built atop of kubernetes. Not sure if versions of those two will be in sync forever but for the time being supported kubernetes and OpenShift API versions do match.

I'm planning to go through most interesting API calls in another post. So far you can try looking at existing documentation:

https://docs.openshift.org/latest/rest_api/openshift_v1.html - OpenShift Rest API documentation
https://docs.openshift.org/latest/rest_api/kubernetes_v1.html - Kubernetes API documentation
https://docs.openshift.org/latest/architecture/additional_concepts/authentication.html - authentication documentation
https://docs.openshift.com/enterprise/3.0/architecture/additional_concepts/other_api_objects.html - you can read here about OAuth objects

Update (June 2016): You can run `oc ... --loglevel=8` to see exact calls made by the oc utility. It's priceless to understand how an API call works. Keep in mind the output exposes your token.

Thanks a lot to @liggit, without whom I wouldn't be able to write the above!

Monday, December 15, 2014

Firefox OS on Mozilla developers' phone Firefox Flame Review

This is my subjective view on the Mozilla Firefox flame device running Firefox OS 2.x nightlies. I've been using it as my main phone device for maybe 3 months already and I think that's enough time so that any critique is not caused by being used to other devices but real indication of user experience.

First I have a nice impression from the design of the phone. It's really nice to hold it and look at it.

Also I'm very impressed with the battery life. With lite usage it works for me maybe 5 days long in the city with good network coverage.

It has a very sane user interface that I immediately got used to.

But here are the pain points I see.

Foremost is the presence of only one button. Having used android before I would say that I still often wish I had the back button and the menu/settings button. Why lose the real estate of the device with putting only one button and all this empty space around?
I see full screen apps usually put a tiny arrow at the bottom of the screen but it is hard to spot and hard to tap on with the touchscreen. Back button is often missing in apps and it's hard to go back (one has to resort to the home button and then navigate back to the desired place.
If somebody from mozilla cares, then please reconsider adding the other two buttons.

The OS has a very nice feature - hold home button and you can select applications and close them (unlike android where you can't close an app). There is a lot of space on the screen but the X-button is so tiny that I always have troubles tapping on it (maybe I should file a bug report for that).

Another nifty feature is sliding from one app to another. But this feature interferes with sliding feature of app itself. For example when I try to view previous/next image in galery app, I often change the app itself. IMO would be better to switch apps only when sliding on particular location of the screen like the top 1/3 or 1/4 part of the screen. Because in an app one would rarely need to slide that high.

Another nice feature is holding home button to see list of apps and have ability to close apps. Very nice if the "x" was a little larger so I can easily tap on it.

Time... very strange bu my phone often hangs. I reboot it but them time goes totally wrong. I need to disable/reenable automatic time for it to be resynced. Very confusing at times.

Contacts:
Not sure why every time I try to sync contacts with google, I need to enter password again and the two factor code. Sync is only one way also.
Renaming of contact is not detected and I end up with two contacts. Finding duplicates didn't catch that as well.
Also too bad the contact app does not show anything within braces like John (the good boy). I see only John in the list and I can't differentiate between the Johns until I open the contact.
Another thing is that app does not search through all fields in contacts. I often have Name and phonetic name of contacts. It's too bad I have to search twice - with native name and phonetic name to find a contact.
One other contact deficiency is that it only downloads contacts from "my contact". I used to have some other groups to select what I want inside the phone but had to put all contacts under "My contacts" so they are synced to the phone.

email:
lack of categories and lack of ability to do anything with attachments whatsoever. Not with the app and not by accessing gmail with browser. Actually accessing through browser was mostly broken last time I've tried.

There's some issue with charging battery over USB from a standby laptop. It works with an android phone... I guess that might be a HW quirk.

Keyboard:
I'd only say rather inconvenient compared to android.

It turned out into a rant but really Firefox OS needs a lot of polish until it's a pleasure to work with. So far I'm still with it for the foreseeable future.

Tuesday, June 10, 2014

Git - see what added so much space to the repo

Here's a good way to see what added space to the git repo recently.

$ git log --all --pretty='%h' --since='{2014-06-06}' | xargs -n1 sh size.sh
total: 240 0.0MB for 6df23f5
total: 88709989 84.6MB for f66712d
total: 26473 0.0MB for c397131
total: 462 0.0MB for 7c78e1c

And here is the size.sh file:

#!/bin/bash

if [[ -z "$1" ]] || [[ "$1" == "-h" ]] || [[ "$1" == "--help" ]] ; then
    echo "Get change of size introduced by individual commits"
    echo "Usage : $(basename "$0") <commit> ..."
    echo "Example: git log --all --pretty='%h' --since='{2014-06-06}' | xargs bash size.sh"
    exit 1
fi

while [[ "$1" ]]; do 
  commit="$1"
  shift
  total=0
  git diff-tree -r $commit^1 $commit | {
    while read -r sperms dperms oldobj obj flag filename; do
      case $flag in
        A) bytes=`git cat-file -s $obj`
           ;;
        D) bytes=-`git cat-file -s $oldobj`
           ;;
        M) bytes=$(( `git cat-file -s $obj` - `git cat-file -s $oldobj` ))
           ;;
        *)
          echo "ERROR: unknown flag \"$flag\" in in commit \"$commit\": $sperms $dperms $oldobj $obj $flag $filename"
          continue
          ;;
      esac
      total=$(( total + bytes ))
    done
    total_mb=`echo $total | awk '{printf "%.1f",$1/1024/1024}'`
    echo "total: $total ${total_mb}MB for $commit"
  }
done

credits to for the idea:
http://stackoverflow.com/a/10847242/520567