How to use dbgsrv, the Process Server

Posted by on . No comments.

If you’ve ever used Debugging Tools for Windows, you’ve probably used remote debugging. Users of windbg/ntsd are usually familiar with the “.server” command and connecting to remote debugging sessions using “Connect to Remote Session” in windbg. Knowing this, what good is the dbgsrv utility that comes with the Debugging Tools for Windows? How is it different from a normal remote debugging session?

For anyone familiar with dbgsrv, it is similar in purpose and function to msvsmon, the visual studio remote debugging monitor. Both applications are “process servers” for their respective debugging environment. Running dbgsrv on a remote machine allows developers to attach to any process on that machine, or launch processes under the debugger.

Using dbgsrv is slightly mysterious, as the UI for using it is a bit cryptic in WinDbg. I’m going to run through a few examples to show how useful dbgsrv can be.

To start, we need to get dbgsrv running on the machine you want to debug. Usually, this will be a test computer that you want to debug from your development computer. If you want to follow along the examples, you can use a single machine, although this scenario isn’t very useful for real world debugging. To start dbgsrv, run it from the command line giving it parameters to pick a transport to use. Usually, tcp is the easiest when not working with domain-joined machines:

dbgsrv -t tcp:port=31337

If there are no errors, dbgsrv will start silently and start listening for connections. (You may see a firewall dialog at this point). To use the process server from windbg, use the “Connect to Remote Stub…” menu item from the File menu:

Screenshot of Connect to Remote Stub in Windbg

The connection string should match what you’ve started dbgsrv with. For instance, if dbgsrv is running on my computer named “timtst”, I would connect to “tcp:port=31337,server=timtst”. After accepting the connection string in this dialog, there will be no indication of failure or success until you actually try to use it. Once the connection string is entered, you can try to attach to a process using File->Attach to Process, or using F6. If the connection was successful, you will see a list of processes on your test computer, or an error if the connection was not successful.

To use the same functionality in cdb/ntsd, you can use the “premote” command line argument. For instance, to use the same dbgsrv instance as in the previous example to attach to process ID 4000, you can use the following command with ntsd:

ntsd -premote tcp:port=31337,server=timtst -p 4000

Using ntsd also lets you start a process under the debugger through a process server:

ntsd -premote tcp:port=31337,server=timtst C:\MyTestApp.exe

Note that the path given refers to a path on the remote machine running dbgsrv, not the machine running ntsd. This functionality is not available in windbg.

There is an excellent write-up with more information about dbgsrv at nynaeve.net

If you’ve found this interesting, I highly recommend the book Advanced Windows Debugging
by Daniel Pravat. As one of the most informative books about the Windows Debugging toolset, it has a host of information about dbgsrv and the rest of the tools.

Travelling down a stack of dependency woes – How to parse HTML in Windows with Python

Posted by on . No comments.

I was hoping I could parse HTML in Python in Windows. As it turned out, every step I tried ended up leading to another step. In case you are about to lose an entire day dealing with all these steps, I wrote them here.

  1. Problem 1: Beautiful Soup isn’t supported anymore
    Beautiful Soup is the de facto HTML parser. Beloved by Python programmers, it’s capable of dealing with broken and messy HTML. Sadly, the libraries that it used are being replaced, and the main developer doesn’t have time to work on it anymore.
    Solution: This was the easiest problem to deal with. I asked the New York Python Meetup, and they all recommended lxml.
  2. Problem 2: lxml doesn’t have a Python 2.7 build
    The easy solution – “easy_install lxml” – is supposed to get an egg file precompiled with lxml’s dependencies (at least, says the INSTALL file in the download).
    There were two problems:
    1. It doesn’t
    2. None of the .exes on the site are for Python 2.7.
    Solution: As it turns out, there’s a way around this dilemma; someone’s posted a script to build it online. It’s only for 32-bit though, it seems, but I gave it a short spin anyway.
  3. Problem 3: Cython error
    Solution: This fix was easy.
  4. Problem 4: “vcvarsall.bat” missing
    As it turns out, building many Python packages requires vcvarsall.bat, which is probably a compiling tool of some kind in Microsoft’s toolchain. The fix that comes up in search engine results involve hacking in a different compiler (gcc from MinGW), which I suspected might cause other incompatibilities.
    Solution: After talking with a friend from Microsoft, I determined that downloading and installing the Windows SDK would be a good place to start. Though that didn’t work, I did end up installing Visual Studio C++ Express, which did include vcvarsall.bat.
  5. Problem 5: “vcvarsall.bat” missing
    For some unknown reason, even after adding vcvarsall to the path,  the error still came up.
    Solution: It was at this point that I realized that the build script was for 32-bit. If I was going to go through the trouble of trying it, maybe it would be worth trying a 32-bit precompiled exe, which I ended up discovering on the same site I visited earlier.
  6. Problem 6: Installation didn’t work
    Even though the install went find and “import lxml” worked without a hitch, the lxml package was strangely empty – there was nothing in it!
    Solution: I went through site-packages and cleaned it out – there were two separate lxmls in there from my previous experiments. Removing one of them cleared it all up.

(I recently had an experience where I couldn’t beta test some software because it was built for 32-bit, but my computer was 64-bit. As it turned out, that didn’t apply here.)

Possible lesson: If something is unlikely to work, but is easy and quick, try it anyway.