quick debugging KVM VM issues

See a hang or infinite loop, or perf issue with VM on KVM? Here's how to get a trace of it so a bugzilla report can be meaningful:

First attach VM configuration XML. That is obtained by:

>  sudo virsh dumpxml [vm_name] > some_file

Cole Robinson wrote on 09/23/2014 04:24 PM:
> sudo debuginfo-install qemu-system-x86
>
> Then on the next hang, grab the pid of the busted VM from ps axwww, and do:
>
> sudo pstack $pid
>
> The dump that output in a bug report, along with
> /var/log/libvirt/qemu/$vmname.log. File it against qemu

Also interesting might be system log from Host and guest. On Fedora you can obtain it by a command similar to:

> sudo journalctl --system --since today

replicating HTTP Server replies using ncat and socat

I was looking at an issue that rest-client ruby gem raised an error on `#cookies_jar` on one particular server while it worked fine on a couple of public servers I tried [1].

I was just going to write a simple script to serve as a HTTP server to return me same response as the offending HTTP server but hey, I thought, there must be an easier way.

So I just obtained raw response from original server, put it into a file and asked netcat to listen and give it back on request.

$ cat > response.raw << "EOF"
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 36
Content-Type: text/html; charset=utf-8
Last-Modified: Mon, 11 Apr 2016 05:39:53 GMT
Server: Caddy
Date: Tue, 10 May 2016 08:10:17 GMT
Set-Cookie: OPENSHIFT_x7xn3_service-unsecure_SERVERID=c72192d7fe9c33d8dec083448dd4f40f; path=/; HttpOnly
Cache-control: private

Hello-OpenShift-Path-Test http-8080

EOF

$ nc -l 8080 < response.raw

## on another console

$ curl -v localhost:8080 

That's the simplest I could get. It will return the same thing regardless of path and query string you put in your client URL. e.g. this will work the same:

$ curl -v localhost:8080/path&asd=5

Now if you want your server to return something multiple times, then you can try

$ nc -kl 8080 -c 'cat response.raw'

Another option if your system lacks netcat is the `socat` utility.

$ socat TCP-LISTEN:8080,fork EXEC:"cat response.raw" 

If you remove `fork` from the options, it will exit after first connection served. But we can also listen over HTTPS:

$ socat OPENSSL-LISTEN:8080,cert=/path/cert.pem,verify=0 EXEC:"cat response.raw"

Again, add `fork` option to keep listening. This above will ignore client certificate. In fact you can create proper client cert and configure SSL verify. But that's beyond today's topic. FYI, use `socat` version 1.7.3.1+, otherwise you'd be hit with weak DH key used [2]. As a workaround you could generate DH key in a file and provide it with the `dhparams` option to socat.

[1] https://github.com/rest-client/rest-client/issues/487
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1021946