1. Problem 1: Beautiful Soup isn’t supported anymore
   Beautiful Soup is the de facto HTML parser. Beloved by Python programmers, it’s capable of dealing with broken and messy HTML. Sadly, the libraries that it used are being replaced, and the main developer doesn’t have time to work on it anymore.
   Solution: This was the easiest problem to deal with. I asked the New York Python Meetup, and they all recommended lxml.
2. Problem 2: lxml doesn’t have a Python 2.7 build
   The easy solution – “easy_install lxml” – is supposed to get an egg file precompiled with lxml’s dependencies (at least, says the INSTALL file in the download).
   There were two problems:
   1. It doesn’t
   2. None of the .exes on the site are for Python 2.7.
   Solution: As it turns out, there’s a way around this dilemma; someone’s posted a script to build it online. It’s only for 32-bit though, it seems, but I gave it a short spin anyway.
3. Problem 3: Cython error
   Solution: This fix was easy.
4. Problem 4: “vcvarsall.bat” missing
   As it turns out, building many Python packages requires vcvarsall.bat, which is probably a compiling tool of some kind in Microsoft’s toolchain. The fix that comes up in search engine results involve hacking in a different compiler (gcc from MinGW), which I suspected might cause other incompatibilities.
   Solution: After talking with a friend from Microsoft, I determined that downloading and installing the Windows SDK would be a good place to start. Though that didn’t work, I did end up installing Visual Studio C++ Express, which did include vcvarsall.bat.
5. Problem 5: “vcvarsall.bat” missing
   For some unknown reason, even after adding vcvarsall to the path,  the error still came up.
   Solution: It was at this point that I realized that the build script was for 32-bit. If I was going to go through the trouble of trying it, maybe it would be worth trying a 32-bit precompiled exe, which I ended up discovering on the same site I visited earlier.
6. Problem 6: Installation didn’t work
   Even though the install went find and “import lxml” worked without a hitch, the lxml package was strangely empty – there was nothing in it!
   Solution: I went through site-packages and cleaned it out – there were two separate lxmls in there from my previous experiments. Removing one of them cleared it all up.

(I recently had an experience where I couldn’t beta test some software because it was built for 32-bit, but my computer was 64-bit. As it turned out, that didn’t apply here.)

Possible lesson: If something is unlikely to work, but is easy and quick, try it anyway.

Programming languages are great because they make it easy to write code – in a way that makes it easier for us to understand.

In the end though, all of these abstractions were not written with your problem in mind, but a generic, off-the-shelf problem that you don’t actually have.

Thus, write with any language you want. But write in a way that’s appropriate to your problem, and write in a way that’s appropriate to you.

Preconditions:

• Familiarity with writing and compiling a program in Java.
• Not for people who have had a significant amount of experience in commercial programming.

So, you want to write a program in Java? Well that’s easy, I’ll write one right now:

public class MyProgram
{
public static void main(String args[])
{
System.out.println("Hello, world!");
}
}

file: MyProgram.java

That’s nice, you say, but I want my program to do more than that. I want it to be a calendar program. It’ll record appointments and let you keep track of them, as well as remind you which ones are coming up today. So how do we accomplish this?

Now, the easiest way to finish this job is to “just write the program.” The only problem is that we’re not doing it in English. It would, in fact, be nice if we could just write the program in English, and then have someone else translate it for us:

[Originally written on 10/22/2005 at 8:09 pm]

[12/7: I decided this isn't really worth writing yet. - C.L.]

Given a 64-bit integer, what is the fastest way to reverse the ordering of the bits? The naive approach is to use a loop getting 1 bit into place each iteration. The result is something like this:

private static ulong RevBits1(ulong p)
{
ulong r = 0;
for (int i = 0; i < 64; i++)
{
r = r >> 1;
r = r | (p & 0x8000000000000000);
p = p << 1;
}
return r;
}

If you don’t count the loop operations, this ends up with 256 operations, 4 per iteration. If you unroll this, you can make one of the terms a constant factor, bringing the operation count down to 192. We can improve on this with a divide and conquer approach:

private static ulong RevBits(ulong p)
{
p = (p << 32) | (p >> 32);
p = ((p << 16) & 0xFFFF0000FFFF0000) | ((p >> 16) & 0x0000FFFF0000FFFF);
p = ((p << 8 ) & 0xFF00FF00FF00FF00) | ((p >> 8 ) & 0x00FF00FF00FF00FF);
p = ((p << 4) & 0xF0F0F0F0F0F0F0F0) | ((p >> 4) & 0x0F0F0F0F0F0F0F0F);
p = ((p << 2) & 0xCCCCCCCCCCCCCCCC) | ((p >> 2) & 0x3333333333333333);
p = ((p << 1) & 0xAAAAAAAAAAAAAAAA) | ((p >> 1) & 0x5555555555555555);
return p;
}
Harder to read, but this version is only 28 operations. In asm we could get this down even farther on 64 bit machines, since the first statement can be written as a 32 bit rotation, and the second statement can be written as two 16 bit rotations, bringing our operation count down to 24. Since each of our other operations is nearly equivalent to an opcode (on 64-bit processors at least) this is pretty close to the actual number of opcodes that could be generated by a good optimizing compiler.

It would be interesting to see if this could be optimized further

-Tim

My version: Build every one to throw away.

Fred Brooks’ classic The Mythical Man-Month made this observation developing a new piece of software – inevitably, many of the problems that will need to be solved won’t be known until you try to solve them. The first try will incorporate many feelings of should-have and would-have that will be fixed on the second try.

This is much like a journeyman craftsman building his or her first bench, or planting their first garden. The first try is a learning experience. There is no way around this, nor should there be – learning is often best by doing. The valuable asset is the experience of the builder, not the product.

But we don’t live in the Middle Ages, the age of guilds. We live in the post-Industrial Age. Factory owners don’t consider hoarding all the toys or cars or electronics they produce as an asset. Henry Ford’s genius was to redesign not just the golden eggs, but the machine that lays them.

As programmers, we are not working on a code base. We are working on a machine that produces code. The machine is made up of us, our experiences, the tools we build to make code (which can be made out of code themselves!). This is where results come from, and we should spend our time tuning this machine by producing more, not protecting what we have.

Does this mean that we should throw out old code? By no means! Old code is one of the most efficient resources we have for producing new code. But every process of manufacture in the past has been made into a more automatic and refined process. How could we consider ourselves any different?

Does this mean that we should code like crap? By no means! As was once said, “If you write the first one to throw out, you will end up throwing out the second one as well.” The point is, write good code, but be willing to write new code – the point is to make not good code, but a good code factory.

[more (full article)]

I was just as much shocked as dismayed. Mulberry was hawked by so many colleges that I assumed its destiny was all but sealed. College students, as early adopters, would all be exposed to Mulberry and see how wonderful it was, and they’d take it to their jobs, promoting an almost viral spread.

I guess the Thunderbird/Outlook duopoly was just too strong for Cyrusoft to handle. However, all is not lost, as Mulberry is available for free now.

Let’s take a look at all the wonderful features of this program! (full article)

[originally started a long time ago]

Security is not just about locking down a system from a list of attacks. The way you design a UI dictates how people behave, and people’s behavior is responsible for 90% of attacks. [citation needed] Defaults matter. No one forgets to lock an automatically locking door.

The fact is, while a security system must be set up to prevent hacking attacks, guesswork, and theft, it must also be designed in such a way that leads people to behave more securely. When you have a system where keys are hard to create but easy to copy, naturally, people will end up using the same keys at eBay that they do at Flickr.

The need to “educate users” is an indication of design failure.

How could I justify something so outlandish? Simple. The difficulty of a change (how many hiccups, how many bugs) depends not so much on its scope as it does on how well planned it is. Inevitably, we underestimate how much planning small changes take, and overestimate when we see big, scary changes. Does this mean rewrite is easy? Of course not. But it’s not impossible, or as costly as we think, and just the opposite is true for minor changes.

*The usual caveats: If you have a problem that can be solved with a small change, I recommend sticking to that before resorting to bigger changes. But once you start seeing so-called small changes multiply, you may have hit the stall point.

Most people intuitively know when they’ve gotten stuck. Suddenly, after plowing through mountains of work, returns suddenly diminish dramatically. People who program late at night will recognize this phenomena; coding turns from an art to a masturbatory exercise in frustration. (This also comes up when doing late night work as a student, interestingly.)

The reason people can overlook the stall point is that it’s not one point. If we got pitched headlong into a freezing room, we’d be far less likely to leave the thermostat alone.

Being acutely sensitive to even the smallest amounts of frustration is a good way to pick up on incoming stall points.

Be intolerant of annoyances.

http://pages.cs.wisc.edu/~driscoll/misc/aqsis_mystery/