random bits of computer info: 2015

Thursday, November 26, 2015

redsocks on Fedora/Red Hat Enterprise Linux/CentOS

There is an amazing piece of software - redsocks. What is basically does is listen for TCP (also has some UDP and DNS support), and if anything comes its way, it redirects through a SOCKS4/5 server or HTTP Connect/Relay proxy.

On linux all one needs is to make iptables DNAT desired connections to the ip/port the redsocks daemon is listening.

My only issue was that the redsocks package not available under Fedora/Red Hat Enterprise Linux. I figured debian has a package with systemctl integration so thought to leverage that work. But Debian GNU Linux uses .deb package format, while fedora uses .rpm.

Fedora comes with the `alien` package that can convert between deb an rpm. So I downloaded the deb from Debian testing and gave it a try:

$ sudo alien -r redsocks_0.4+dfsg-2_amd64.deb
Warning: Skipping conversion of scripts in package redsocks: postinst postrm prerm
Warning: Use the --scripts parameter to include the scripts.
redsocks-0.4+dfsg-3.x86_64.rpm generated
$ ls
redsocks_0.4+dfsg-2_amd64.deb redsocks-0.4+dfsg-3.x86_64.rpm
$ yum install redsocks-0.4+dfsg-3.x86_64.rpm
<...>
Transaction check error:
file / from install of redsocks-0.4+dfsg-3.x86_64 conflicts with file from package filesystem-3.2-20.el7.x86_64
file /lib from install of redsocks-0.4+dfsg-3.x86_64 conflicts with file from package filesystem-3.2-20.el7.x86_64
file /usr/sbin from install of redsocks-0.4+dfsg-3.x86_64 conflicts with file from package filesystem-3.2-20.el7.x86_64
file /etc/init.d from install of redsocks-0.4+dfsg-3.x86_64 conflicts with file from package chkconfig-1.3.61-5.el7.x86_64
<...>

Dang! `alien` got libevent dependency just right but wants to take ownership of a couple of dirs that should not be generally touched. After some searching I found the solution - install `rpmrebuild` to remove the unnecessary dirs that conflict. Don't worry, actual files are not removed. Only the directory entries inside the RPM content.

$ rpmrebuild -pe redsocks-0.4+dfsg-3.x86_64.rpm
Do you want to continue ? (y/N) y
result: /home/user/rpmbuild/RPMS/x86_64/redsocks-0.4+dfsg-3.x86_64.rpm

`rpmrebuild` in this way will present you with a text editor. Find '%files' section and remove all directory entries not specific to redsocks (i.e. "/", "/lib", ...).

Finally create the user that will daemon run as (part of debian postinstall script):

$ sudo adduser --system --home /var/run/redsocks --no-create-home -U redsocks

And that's all. Now do `yum install /home/user/rpmbuild/RPMS/x86_64/redsocks-0.4+dfsg-3.x86_64.rpm` and that should be it. Configuration can be found in /etc/redsocks.conf and `systemctl enable redsocks; systemctl start redsocks`.

Monday, September 14, 2015

Copy files to running OpenShift 3 pods

Needed to copy a newer mysql client for debugging purposes into an OpenShift v3 pod. /fortunately the mysql client binary is self contained and does not need special shared libs; worked with v5.6.26/

Here's how I managed to do so:

oc exec -i [pod name] -n [project] -- bash -c 'cat > [remote file]' < [local file]

e.g.
oc exec -i mysql-55-centos7-1-f42js -n 0wwwv -- bash -c 'cat > /tmp/mysql' < /tmp/mysql

What does it do?

Invokes `exec` command with `-i` option so that stdin is passed (otherwise stdin is ignored). The remote command cats everything into desired path. You need to `chmod 755 [file]` inside the pod if that file is to be executed.

To get into the pod in a shell try `oc rsh`.

Tuesday, July 21, 2015

HTTP Proxy vs overriding hostname

There's known trick with `curl` to connect a server via IP address and override the virtual host you connect to using the `Host:` header. Or using the `--resolve` option. This is mainly useful to try test server instances that don't have proper DNS records set.

curl -H "Host: abc.example.com" 8.7.6.5
curl --resolve abc.example.com:80:8.7.6.5 http://abc.example.com

This is quiet handy. And in unfortunate circumstances one can need that but behind a proxy. So I was banging my head to understand how to do it. Eventually I resorted to looking at rfc2068.

The news are not good (as much as probably very little people care) - there is no way to perform that hack over a standard HTTP Proxy. Here's and excerpt:

5.2 The Resource Identified by a Request

   HTTP/1.1 origin servers SHOULD be aware that the exact resource
   identified by an Internet request is determined by examining both the
   Request-URI and the Host header field.

   An origin server that does not allow resources to differ by the
   requested host MAY ignore the Host header field value. (But see
   section 19.5.1 for other requirements on Host support in HTTP/1.1.)

   An origin server that does differentiate resources based on the host
   requested (sometimes referred to as virtual hosts or vanity
   hostnames) MUST use the following rules for determining the requested
   resource on an HTTP/1.1 request:

     1. If Request-URI is an absoluteURI, the host is part of the
        Request-URI. Any Host header field value in the request MUST be
        ignored.

     2. If the Request-URI is not an absoluteURI, and the request
        includes a Host header field, the host is determined by the Host
        header field value.

     3. If the host as determined by rule 1 or 2 is not a valid host on
        the server, the response MUST be a 400 (Bad Request) error
        message.

   Recipients of an HTTP/1.0 request that lacks a Host header field MAY
   attempt to use heuristics (e.g., examination of the URI path for
   something unique to a particular host) in order to determine what
   exact resource is being requested.

There are two ways to ask a HTTP proxy to perform a request:

GET http://abc.ecample.com/ HTTP 1.0

GET / HTTP 1.0
Host: abc.ecample.com

In the first form (point #1 above), even if `Host` header is provided, the proxy server must ignore it and send to the target web server whatever host is provided in the absolute URL. That's why it's technically impossible to do this over a HTTP proxy.

This doesn't mean you can't do it with a socks server or if proxy allows CONNECT to the desired destination port.

With socks perhaps `curl` should do just fine (haven't tested). Using CONNECT to port 80 though would require manual interaction with the socket or a little programming as standard tools would not use CONNECT for simple non-encrypted HTTP connections.

Premium content:
Here's how to use proxy with auth via telnet:
http://unix.stackexchange.com/a/36632/14907

And here's a good example how to use the CONNECT method manually:
http://blog.vivekjishtu.com/2013/09/using-connect-method-on-http-proxy.html

How to get certificate chain by ruby

Looking on the Internet I couldn't find any examples how to get a hold on a remote server cerificate chain. But it turned out to be rather simple looking at ruby 2.2.2 api docs.

https://gist.github.com/akostadinov/fa54e7bc5a1858af5677

Thought might be useful to somebody to post in public..

Thursday, July 9, 2015

Apache HTTPd SSL/OCSP primer

I just stumbled over some old howto I used for testing OSCP. Here I'm pasting it to hopefully help somebody understand how it works. It assumes you have already up and running Apache HTTPd web server.

Create needed certificates

Download this useful script:
https://git.fedorahosted.org/cgit/pkinit-nss.git/plain/doc/openssl/make-certs.sh
Use the script to generate certifictes:

./make-certs.sh europa.sfo.corp.google.com test@example.com all ocsp:http://europa.sfo.corp.google.com/
./make-certs.sh america.sfo.corp.google.com test@example.com all ocsp:http://europa.sfo.corp.google.com/

Prepare and start responder using openSSL

Create index.txt file which contains information about certificates the responder is handling.
File is plain text file with fields separated by TAB.
Here is example of structure:

V       100320100000Z           593C5290F246444B        unknown DC=com, DC=example/mail=test@example.com, CN=europa.sfo.corp.google.com
V                       B9290C71D224ACB3        unknown DC=com, DC=example, CN=Test Certifying CA
R       131021200751Z   100324142709Z,superseded        593C5290F246444C        unknown DC=com, DC=example/mail=test@example.com, CN=america.sfo.corp.google.com

The columns are defined as:
#define DB_type         0 /* Status of the certificate */
#define DB_exp_date     1 /* Expiry date */
#define DB_rev_date     2 /* Revocation date */
#define DB_serial       3       /* Serial No., index - unique */
#define DB_file         4      
#define DB_name         5       /* DN, index - unique when active and not disabled */

Notes:
- DB_type could be only V | R, E is not working
- date format is YYMMDDHHMMSSZ
- DB_serial has to be in HEX with upper case letters
- use example provided above, just don't forget to change serial numbers

Start responder using:

openssl ocsp -index index.txt -port 8088 -rsigner certs/ca.pem -CA certs/ca.pem -text

Check if responder is working fine using:

openssl ocsp -issuer certs/ca.pem -CAfile certs/ca.pem -url http://localhost:8088 -cert certs/europa.sfo.corp.google.com.pem

should return GOOD certificate status

openssl ocsp -issuer certs/ca.pem -CAfile certs/ca.pem -url http://localhost:8088 -cert certs/america.sfo.corp.google.com.pem

should return REVOKED ... reason: superseded

Modify $EWS_HOME/httpd/conf.d/ssl.conf:

- add this to the end of file:

#OCSP
SSLVerifyClient on
SSLVerifyDepth 10
SSLOCSPEnable on
SSLOCSPDefaultResponder http://localhost:8088/
SSLOCSPOverrideResponder on

- httpd has to ask for client certificate and enable OCSP and set default responder

- uncomment SSLCACertificateFile /etc/pki/tls/cert.pem line in the SSL configuration file and make sure your new CA is added to the cert.pem file
or just point this to oscp/certs/ca.crt (generated in one of previous steps)

- copy some static content to $EWS_HOME/httpd/www/html/
e.g. echo "OCSPTestSucceed" > $EWS_HOME/httpd/www/html/ocsp.txt

- start apache

Now try it out:

run:

wget --output-document=/dev/null --no-check-certificat --certificate=client_cert/america.sfo.corp.google.com.pem --ca-certificate=client_cert/ca.pem  https://<your server>/ocsp.txt

result:

Resolving <your server>... 10.34.34.43
Connecting to <your server>|10.34.34.43|:443... connected.
OpenSSL: error:14094414:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate revoked
Unable to establish SSL connection.

run:

wget --output-document=/dev/null --no-check-certificat --certificate=client_cert/europa.sfo.corp.google.com.pem --ca-certificate=client_cert/ca.pem  https://<your server>/ocsp.txt

result:

HTTP request sent, awaiting response... 200 OK

References

Attribution

Thanks to my colleague Rajesh for preparing initial info.

Wednesday, July 8, 2015

OpenShift v3 REST API usage

Time for a quick v3 api trial. The REST api is completely changed. First notable thing is authentication. Basic auth is no longer supported. Now the only supported auth types are oauth token and client SSL certificate.

Lets see how we can get an oauth token. First method is to go with a browser to https://your.openshift.master.server.example.com:8443/oauth/token/request

That will present you with a nice page explaining you how to use your newly acquired token. But we want to perform this in a more machine friendly manner so let's do with openshift-challenging-client via curl:

curl -u joe -kv -H "X-CSRF-Token: xxx" 'https://master.cluster.local:8443/oauth/authorize?client_id=openshift-challenging-client&response_type=token'

Keys here are:

use client_id=openshift-challenging-client, otherwise 400 Bad Request is returned
use response_type=token
set X-CSRF-Token header to some non-empty value, otherwise error is returned
actual token is returned in the Location header of the 302 response per the OAuth spec as access_token=VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU

See partial curl output:

< HTTP/1.1 302 Found
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Expires: Fri, 01 Jan 1990 00:00:00 GMT
< Location: https://master.cluster.local:8443/oauth/token/display#access_token=VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU&expires_in=86400&token_type=bearer
< Pragma: no-cache
< Set-Cookie: ssn=MTQzNjM3NzI4NXxDSkxSTl8yb0ZjUmZaSDZwNG51UjNDZEx1M29xRldQNGtGZTMwbnhfYlNRV2FuVmYxVHlKSWhWazVKWjR2RDc3X056ZVpqZXl6VWN4T0Nqc1dyX01raDhiUlNSdXFpdkhDalAwWDQzNWdyWExlTmNTUURjN3pQeW9HT1RpVmRtQ1JBPT18qi62Db0PolIHaMmAjtdKPejhCGRY-EUEruT6W_Du2bg=; Path=/; Expires=Wed, 08 Jul 2015 18:41:25 UTC; Max-Age=3600; HttpOnly; Secure
< Date: Wed, 08 Jul 2015 17:41:25 GMT
< Content-Length: 0
< Content-Type: text/plain; charset=utf-8

Now lets perform a real API call to remove the token we just obtained:

curl -vk -H "Authorization: Bearer VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU" https://master.cluster.local:8443/oapi/v1/oauthaccesstokens/VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU -X DELETE

Well done!

Notice how we pass the token - using header:

"Authorization: Bearer VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU"

You might have noticed API endpoint is /oapi/v1/something, where "v1" is the version of the API. How do we get server supported API versions? 2 ways so far:

$ curl -k https://master.cluster.local:8443/
{
"paths": [
    "/api",
    "/api/v1beta3",
    "/api/v1",
    "/controllers",
    "/healthz",
    "/healthz/ping",
    "/logs/",
    "/metrics",
    "/ready",
    "/osapi",
    "/osapi/v1beta3",
    "/oapi",
    "/oapi/v1",
    "/swaggerapi/"
]

Looking specifically at kubernetes API endpoint:

$ curl -vk https://master.cluster.local:8443/api
{
"versions": [
"v1beta3",
"v1"
]

Well forgot to mention. OpenShift v3 does give you access to the plain kubernetes REST API as well to the OpenShift REST API, because OpenShift is built atop of kubernetes. Not sure if versions of those two will be in sync forever but for the time being supported kubernetes and OpenShift API versions do match.

I'm planning to go through most interesting API calls in another post. So far you can try looking at existing documentation:

https://docs.openshift.org/latest/rest_api/openshift_v1.html - OpenShift Rest API documentation
https://docs.openshift.org/latest/rest_api/kubernetes_v1.html - Kubernetes API documentation
https://docs.openshift.org/latest/architecture/additional_concepts/authentication.html - authentication documentation
https://docs.openshift.com/enterprise/3.0/architecture/additional_concepts/other_api_objects.html - you can read here about OAuth objects

Update (June 2016): You can run `oc ... --loglevel=8` to see exact calls made by the oc utility. It's priceless to understand how an API call works. Keep in mind the output exposes your token.

Thanks a lot to @liggit, without whom I wouldn't be able to write the above!