Saturday 26 July 2014
The other day, a coworker mentioned that he wasn’t using high resolution timers on Windows because he heard “they caused clock drift”. His statement struck me as odd. Why would checking the time change the time? Are we causing some sort of quantum observer effect? So I asked him: why don’t we just check what the function does?
True, we can’t just open up the source files, but we have something almost as good: the binary itself! A short C function almost always translates into a short assembly function, and GetTickCount is no different. To follow along at home, fire up windbg attached to a 64 bit process (any will do, like notepad) and run “uf KERNELBASE!GetTickCount”. We see this:
0:000> uf KERNELBASE!GetTickCount
000007ff`14211250 b92003fe7f mov ecx,offset SharedUserData+0x320 (00000000`7ffe0320)
000007ff`14211255 488b09 mov rcx,qword ptr [rcx]
000007ff`14211258 8b04250400fe7f mov eax,dword ptr [SharedUserData+0x4 (00000000`7ffe0004)]
000007ff`1421125f 480fafc1 imul rax,rcx
000007ff`14211263 48c1e818 shr rax,18h
000007ff`14211267 c3 ret
We can analyze this function step by step, but from a quick glance, we see we have an immediate load, two memory reads, a multiply, and a shift. It seems unlikely that this will cause any “clock drift”. So how is it possible that we can read the current tick count so simply? A quick google search reveals that this is the “shared user page”, a page mapped into all usermode processes. All the kernel needs to do is update this value every time a clock interrupt occurs and magically we have a GetTickCount function that is so fast it’s nearly free. And no worries about a clock drift.
So this means we’ve dispelled the myth of the mysterious clock drift, right? Well, not exactly. We know that GetTickCount seems unlikely to ever cause the clock to drift, but it turns out there is a real “clock drift problem”, but that deserves its own blog post.
Share or discuss
Thursday 3 July 2014
Chris + Programming
It’s not actually that hard to automate browser testing! Let’s dive in. We won’t use any obscure or (relatively speaking) bleeding-edge technologies here; just the basics. We’re trying to strip out all the fancy names and get to a nice, minimal, testing setup.
The first question to answer is this: what do we use to test with? You’ve probably heard of Capybara, QUnit, Jasmine, Mocha, Karma, Protractor, WebDriver, Selenium, etc., but which does what? Let’s sort it out.
Regular old JS, do-it-all testing.
Candidates: QUnit, Jasmine, Mocha
If you want to assert that 2+2 is 4 using your fancy new addition framework, these are the usual suspects. QUnit is very old but tried-and-true (jQuery uses it); Jasmine is newer; and Mocha is the newest. From what I’ve heard Mocha has better asynchronous support than Jasmine, so when in doubt, I just use that.
Make the browser work automatically
What they’re good for: Automating web browsing
Yes, Virginia, there is only one actual browser automator, and that’s Selenium! I spent years (well, hours) googling this before I realized what was going on.
Candidates: WebdriverJS, WebDriverJS (I wish I was joking), Nightwatch.js
Cooking it up
Now that we’ve got that sorted out, our path is pretty simple, and we’ll need all three. What we’ll do is:
- Start Selenium in the background; it can automatically spin up Firefox, IE, and Chrome windows on its own like magic.
- Install WebDriverJS so we can talk to Selenium (https://code.google.com/p/selenium/wiki/WebDriverJs)
- Create tests in Mocha that use the WebDriverJS library to automate the browser.
Here we get the best of every world. Selenium, a well tested browser automator, handles making Firefox do spooky automatic clicking. We use the official WebDriverJS to talk to Selenium, and we put our tests, just like any other test, in a standard Mocha example – no need to have some separate, slightly-different-syntax not-invented-here framework just to do our E2E testing.
While I could write a tutorial that would rapidly go out of date, I think it would be better to just link to the latest up to date tutorials and make some notes.
Prerequisites: node, npm, java
- Start by installing Mocha and get the basic test running.
- Download the Selenium server.
- Install WebDriverJs (selenium-webdriver) so you can use Selenium in Node.
- Download and include the proper Selenium drivers for Firefox/IE/Chrome
- Run Selenium with Java, making sure to specify the drivers you downloaded in the previous step.
- Replace the basic test you wrote in Mocha with this one: https://code.google.com/p/selenium/wiki/WebDriverJs#Writing_Tests
- That’s it!
Some caveats I ran into:
- First, while mocha understands being installed locally and globally (npm install vs npm install -g), selenium-webdriver only understands being installed locally.
- Downloading the drivers for FF/IE/Chrome is a bit of a pain since the web links redirect to different places.
Other than that though, just use the standard instructions for Mocha/Selenium/WebDriverJs. No need to remember anything else from a random blog post (AKA, this one).
The only tricky part about autotesting your web browser is deciding what to use. Using Mocha + Selenium with WebDriverJS is a nice, commonly supported combination that’s minimal.
Adding things like Capybara, Nightwatch, and Protractor is nice, but frankly, you’ve got to install Selenium anyway, and you’re probably already using Mocha or something similar for your browser testing. All those new, fancy-name frameworks are doing is adding a bunch of syntactic sugar and extra dependencies. Alternatively, there are plenty of tutorials for setting this up, that require all sorts of caveats and weird tricks to remember. Who needs that?
Share or discuss
Saturday 12 April 2014
Chris + Programming + UI Design
An excellent game about managing inventory. And the occasional dungeon. Via
In some video games, the player has an inventory – a set of virtual items that he/she possesses in the game world. Just like in the real world, virtual space is often limited, and we’re forced to discard and sort.
Some of us are naturally inclined to clean, to discard things until we feel organized again. That natural instinct defends us from asphyxiation under mountains of stuff.
Programmers have an inventory as well. Though that inventory is purely mental, it is no less of a burden to organize. While coding, I’ve squirreled away these nuts in the knotholes of my brain:
Thinking about my programming work as juggling mental inventory leads to several useful ideas:
1) Keep a mental toolkit next to your brain (aka your L2 cache)
If you were a carpenter, you wouldn’t try to hold your hammer, nails, pliers, a saw, and a wrench in your hands all the time – you’d drop something. Similarly, if I tried to remember all the information I needed to do my job, just in my head, I would (a) drop things and (b) get exhausted quickly.
*(As totally obvious as the above sounds, I regularly try to remember far more than my brain can hold, and pay these totally obvious consequences on a regular basis.)
Just like a carpenter needs a toolbox, a programmer needs a info toolkit, pieces of paper or virtual sticky notes, to hold the information we regularly need to do our work and retrieve it quickly. I print out cheat sheets; Gas uses Evernote.
2) Aim to build the IKEA of APIs (or languages)
If you’re building an API for a programmer, they will have to carry extra mental tools to use what you’re building. If you can build the IKEA of APIs, something accessible using simple, readily available tools they already have, your users will be delighted and thankful.
It’s OK to require more tools, especially if your language is going to be 90% of where a user spends their time (SQL comes to mind, though it’s not a perfect example). However, think about how your programmer will hold the tools they need to use your API!
3) Respect those who use your apps
The same goes for app users. Most popular apps/applications do not actually require that much inventory accumulation, but there are quite a few that do. It’s not wrong to require the user to juggle a lot of tools (especially if you want to challenge your users, like a game), but be aware of what you demand!
Inspired by reading 1/3rd of this essay: I Am Overencumbered
Image credit, 2
Share or discuss
Saturday 29 March 2014
Antibiotics are running out. We hear it (quietly) in the news frequently. As antibacterial drugs are used everywhere, superbugs evolve with a frightening resistance to even our newest pharmaceuticals. Stories warn of a future in which even knee surgeries become operations with tremendous risk of death. Someday, they warn, we may miss the decades in which cancer and heart disease were the number one killers, as we regress back to early deaths from TB and E.coli.
The most cost-effective treatment for a disease is not a drug, but a vaccine. The cost to society of polio is only in the thousands now; smallpox is nonexistent. In the modern day however, cost effectiveness makes little sense for a pharmaceutical company. Building a product that will put itself out of business may be good for society, but it’s hardly a survivable way to make a living.
Is there a financial instrument that can properly make transparent the health needs of society? Let’s make up an example.
Case study: disease
Let’s say a city has 10000 people. 20 of them get Disease A. Of those, 10 will die.
If we developed Vaccine A, at a cost of $5, only 2 people would get Disease A, and 1 would die. Assuming those 9 people saved would have added $1 to the economy each, we have now done $4 better than we would have before. However, no one person can pay for developing a vaccine such that it benefits themselves enough (9/10000 x $1) to cover the expense.
One possible financial instrument: Insurance
Let’s say we develop a fund such that anyone who dies of Disease A will get a payout of $2. Someone who buys this instrument has gone from an expected bad outcome of -$1 to $2, and any price with a proper weight under this outcome (9/10000 * $3) would justify spending this amount. Thus, everyone buys into this insurance plan, and the insurer makes, say, 9/10000 * $2.5 * 10000 = $22.5 in premiums.
However, the insurer must pay out 10 * $2 = $20 to the victims of this disease. This means they will only make $2.5 in profit. However, if the insurer takes the $5 expense to develop Vaccine A, they will reduce their required payout by 9 * $2 = $18, and thus make a hefty $15.5 instead.
Would this work? Some of the more obvious challenges to be overcome follow.
Problems with this:
- Moral hazard. What if one insurer takes on this expense, and another insurer, sensing the benefit, offers their own insurance, charging less? Now the second insurer can potentially take the benefits from the first insurer without paying the cost. This is a dangerous proposition in the sense that, once it becomes less cost-effective, then the first insurer will not develop treatment, and both companies will become insolvent.
- Another risk: What if, precisely because the vaccine is being developed, fewer people buy into the insurance plan? You might benefit from having a Kickstarter-like threshold model for purchasing insurance.
- We don’t actually know the proper costs for a vaccine.
Question: Do insurance companies do this already? If so, how?
Could we use new financial instruments to improve society’s health? Which ones do we already use? These are questions that are worthy of investigation. It would be very unsurprising if someone has already done so!
Share or discuss