And most likely, the stuff inside the box is important, but not important enough that you’ll bother going inside the box if you have to work at it. So now you’ll never go inside the box.

Writing a CKEditor Plugin

Exposition: Why should you care about a random text editor?

Modding CKEditor is a well-designed exercise in self-flagellation.

CKEditor is a text editor plugin that works with rich text (bold, italics, tables, links, etc.). If you’ve ever used Gmail or Hotmail, you’ve used something similar.  CKEditor is relatively easy to use, powerful and has lots of features, and it’s used everywhere. But while the code is well written and well documented, at a macro level there are terrible design things going on, and if you want to modify it, you’ll want to shoot yourself shortly thereafter.

Since I’ve been wrestling with it for quite a while now, I wanted to share my thoughts:

• In case you happen to be forced to alter CKEditor as well. Hopefully this will be a useful breadcrumb trail.
• To convince myself I’m not faking it. CKEditor has well written source, so someone could argue “maybe they’re doing it well and you just don’t get it.”
  This is perfectly valid. The best way to figure out if I’m faking it is to actually try and explain why it’s bad, and see if this post turns into “CKEditor is an architectural marvel” in the process.
• Complaining is whining unless you can explain why, and how to make things better.
• I need to understand this stuff myself, and nothing helps with understanding like trying to explain.
• Because I’ve been staring at this for days and need to say something.

Act One: Parallel Universe APIs

Half of CKEditor code essentially replicates JavaScript DOM APIs and jQuery. Let’s use the most radical example of CKEditor manipulation – the dialog architecture.

CKEditor dialogs are used for special properties. If you want to add a link, you would create a dialog to set the URL.

What language do you think dialogs would be designed in? Presumably a language that is used to lay things out in a web browser. Hmmm…what language do we use for that…HT nope, you’re right, it’s JavaScript:

The new dialog system in CKEditor is to be written from scratch. One of the key things in CKEditor is that it doesn’t rely on HTML pages to run. Everything is created on the fly in JavaScript. In this way, we avoid limitations about cross domain serving of the editor code, like CDN solutions, and enhance the editor performance.

http://docs.cksource.com/FCKeditor_3.x/Design_and_Architecture/Dialog_System#Dialog_Definition

We’ll get back to the benefits CKEditor claims later, but let’s think about this. You’re laying out a mini-web page. You’re a web developer and probably lay out web pages all the time. You read A List Apart obsessively; you understand the difference between content (HTML), presentation (CSS), and behavior (JavaScript), and the benefits of separating them.

And then you see this:

(This is actual dialog code that CKEditor uses to show a link dialog.)

Is that HTML? Yup – it looks like a <select> – there’s a list of options (items), and an id, and even a label.
Is that JavaScript? Yup – it’s got an onChange, a setup, and a commit, whatever that is.

This confuzzlement is compounded by the fact that there are about 1000 (literally, see for yourself) lines of this JavaScript/HTML hybrid. If you separated this into HTML and JavaScript, it would be easy to read, not indented six ten tabs in, and it wouldn’t take you an hour of sorting to figure out what a dialog that’s basically a handful of selects and some text fields is doing.

By the way, there’s CSS too:

		var basicCss =
			'background:url(' + CKEDITOR.getUrl( this.path + 'images/anchor.gif' ) + ') no-repeat ' + side + ' center;' +
			'border:1px dotted #00f;';

Let’s establish some of the downsides of this mixed-content architecture:

• Wading a thousand four-hundred line morass of mixed behavior and content every time you want to change things
• No way to test the layout of your dialog without loading all of CKEditor (as opposed to HTML, which can be tested by anyone with a web browser)
• You have to relearn all your HTML/JavaScript/CSS ninja skills in a weird JS hybrid
• Oh and if you miss a brace, it’s not like missing an angle bracket – everything dies. JS is brittle. Did I mention you have to reload all of CKEditor to test your fixes?

So how about the benefits? The CKEditor docs claim that you can do cross domain* loading (for content delivery networks, etc.). While there are plenty of things wrong with the solution, the CDN problem is serious and it would be irresponsible to ignore it. Is there another way we can allow CDN loading without replacing all of our HTML with JavaScript?

One option: Write HTML for the dialogs, but convert it into escaped JavaScript internally. Escaped HTML is ugly, but if we compile it, we needn’t use it except in production.

This rant is just barely getting started, but I’ll stop here for the moment.

Past, Present, Future Vision of AI – Google and AAAI 2011

“And it was fun looking at the comments, because you’d see things like ‘well, I’m throwing in this naive Bayes now, but I’m gonna come back and fix it it up and come up with something better later.’  And the comment would be from 2006. [laughter] And I think what that says is, when you have enough data, sometimes, you don’t have to be too clever about coming up with the best algorithm.”

(Peter’s speaking about search algorithms, but what if you applied this to running a startup? Or life in general?)

From a code perspective, it’s easier to add new features than remove old ones.

From a user perspective, it’s much, much easier to add new features than remove old ones.

Nobody knows about your new feature, but there are at least three people who love that button you added to the toolbar two years ago. Even if no one else cares, they’ll make sure to post angry requests on your forums asking why you got rid of it. If you’re not careful, they’ll accuse your company of “not caring about its users,” or they might even say you “used to be great, but ever since they got successful they’ve started to go downhill”…ever hear that before? [1]

At the same time, it’s important to cut features, or you end up sliding down this curve:

So how do you decide what to cut?

Why Not Just Count?

One of the great things about analytics is getting real data about who’s using what. Every month, Google Analytics sends me an email that tells me exactly what people are reading on this blog. (Hint: It’s not the minesweeper articles)

That gave me an idea: Why not generate similar analytics for a library like jQuery?

There’s a nice list of sites using jQuery available here:

http://docs.jquery.com/Sites_Using_jQuery

Let’s take a site and see what it’s actually using:

(Speaking of wanting to stick to old things, Match.com uses jQuery 1.2…)

Match.com’s landing page is simple enough that we can do an actual list of what calls they make to jQuery, which I’ve uploaded here.

From this, we can tell that the standard selector function $() gets used a lot, as does readCookie and addClass. From there, the next step is to figure out what they’re actually calling.

In the case of jQuery, this is not too hard in principle. Get a version of jQuery, add a line that increments a counter every time a function is used, and inject it into your favorite website. It’s a little too involved for this post, but perhaps in the future (earlier if someone decides to jump in and do it!)

[1] Back in the 90s, Microsoft obsessed about making sure everything from the past worked, even if it mean keeping broken things from earlier versions of Windows:

The Windows testing team is huge and one of their most important responsibilities is guaranteeing that everyone can safely upgrade their operating system, no matter what applications they have installed, and those applications will continue to run, even if those applications do bad things or use undocumented functions or rely on buggy behavior that happens to be buggy in Windows n but is no longer buggy in Windows n+1. In fact if you poke around in the AppCompatibility section of your registry you’ll see a whole list of applications that Windows treats specially, emulating various old bugs and quirky behaviors so they’ll continue to work.

Which would you rather have:

• An app that, based on preferences you enter about cuisine, price, experience, etc., comes up with a list of restaurants in your neighborhood that fit your criteria?
• A friend who’s been to every single restaurant in town that you can always count on to come up with a great suggestion for food, no matter what you’re in the mood for?

With some exceptions (and there are always exceptions), I’d rather go to the person first, then the app. Yelp has plenty of suggestions for coffee and wi-fi in the city, but I still check Cafe Talk. Amazon has great reviews, but I’ll often still look at the page of that one blogger who writes a 20 page article just to talk about the pros and cons of one set of headphones.

Why pick one person’s opinion over the wisdom of crowds and a 200 million dollar company?*

Simple: The person cares, and the person is a person.

*Note: Yelp is actually an exception, to the extent that its full of people. But the people all care about different things.

Products are utilities, services are solutions

First off: this doesn’t apply everywhere.

I don’t need a personal assistant to read me my emails or backup my files. I’m OK with large warehouses of computers doing that, because I know exactly what needs to be done and there isn’t anything surprising or complicated about it. When I know what I need, I look for a utility.

What separates utilities from solutions? Uncertainty.

If I’m looking for a website designer, who would think it was acceptable to hand me a copy of Photoshop, a book on HTML, and say, “here’s your app, go for it?”

A utility is designed to take things we know we need and get them. The best utilities tend to be well targeted and have a specific function. No one has to ask Instagram what he does at a party; it’s pretty ______ obvious.

But even there – how did you get introduced to Instagram?

Did a friend tell you about it? Did you read about it in a blog? Did you see someone using it?

Instagram started out as something unknown, and when it comes to unknowns, we need a personal introduction.

Even if it’s a product, start out with a service

I know the eventual goal of a product-driven startup is to build something that “scales” out–the definition of scale being that we can go out and have a picnic while our software makes monkey.

Nonetheless, I see lots of people who start out with the picnic. Oh, they may be working hard on their product, but there’s no relationship to the customer. They build the system and throw it over the wall and hope it goes viral. Starting a connection to your customers through an impersonal, on-stage launch is how you get casual interest. At the beginning, there will be no one but you to give a personal introduction to your gadget.

It can be as simple as telling the story of what it does and letting them use it, but you have to provide that service the first few times around. Even in the greatest product driven businesses, it’s people who are telling the story, not the product. The eventual goal of scale is not to stop telling the story, but to have others tell it for you.

In most games, there’s a specific role to play. In Serious Sam, I control a wise cracking uber muscled super soldier with absurd amounts of ammunition. In GTA, I’m a city dweller only interested in cars, and perhaps an aspiring criminal lowlife.

When I play FFVI, what am I playing exactly? On the one hand, I’m an explorer, flying around a continent, discovering caves and towns and places, and solving the errant puzzle. I’m also one of a rotating cast of characters, each with their own motivations and personalities, an actor in a vast, world-transforming narrative. On top of that, I’m also playing a mini-tactical combat simulator every 30 seconds.

I’ll bet that most people playing a JRPG play it somewhat more holistically than that. We don’t separate the different parts of the experience unless they break immersion (the Bioshock hacking minigame comes to mind). The trouble with JRPGs is that they’re often so well made that the seams don’t show, but they’re still there. (Now that I think about it, Myst, for all its foibles, is remarkably cohesive for a story game…)

Exercise: What if you actually broke all the different pieces apart? What if there was a game that was completely wandering around a map picking up things and making dialogue choices? And another game that was nothing but fighting through a big arena?

It seems, at least superficially, that both of these games would be much worse than FFVI, but it could just be that we don’t imagine those games polished on the level that FFVI is.

The problem: Why is this d_mn thing taking so long?

Ever since I started working on my own, I’ve spend far more time figuring out how to motivate myself. Because I’m working in a startup, I make sure to do just the minimum amount of work to make a viable product, so that I can save time and work on other things.

And yet, I find myself staring into space, going on Google Reader, Hacker News, playing games, or anything I can find aside from work.

As I’m talking with a friend online, we make some observations:

• While the eventual result is interesting, the work I’m doing is boring.
• The reason I’m bored is because the work is easy.
• Therefore, if I want to be more productive, I need to make my work harder.

I throw out the disorderly, simple, get it done code that everyone says you should write for a prototype. I make things organized. I put all my variables into neat, sensible structures that follow design principles. I use a regular expression to replace 6 lines that I could’ve done by hand in five minutes.

All of a sudden, I’m getting work done again.

What gives?

–

I’ve talked with scores of startups over the years, and they consistently err on the side of overbuilding rather than underbuilding. Fancy prototypes take months to code, months that are wasted if your idea of a customer is the two people you told at the startup meetup you went to last week that said it looked kind of cool; people create entire frameworks to build sites that could be done in the time it takes to write a shell script (that installs WordPress).

Thus, the rule to build fast and build cheap was born.

Am I some kind of unlikely exception? Perhaps. But after giving it more thought, I realized that the need to make some things hard is not surprising, and not unique.

–

Are you working at a startup?

Why?

The pay is (sometimes) lower and the responsibilities are (often) greater. Most of the programmers I know working on their own projects are exceptionally good at programming. The hackers aren’t hanging out at the back of the bell curve. They can pass FizzBuzz, they know it, and that’s exactly why they’re finding a place where they live or die by their skills.

So it’s not entirely surprising that, if you pick people who are fairly competent at coding, so competent that they want to go to a place where they may be solely responsible for the technical success of the product they build, that they might just prefer not to always have simple, somnambulist-friendly code assignments.

Now, as with any essay that argues a point, you can poke holes in this thesis. Perhaps, you might say, the original rule isn’t against giving yourself a technical challenge from time to time, but rather as a safety railing to stop the people who want to make Pandora Craigslist Farmville with an extra layer on top so you can tag your singing cows with pictures from your last missed connection. And that’s a fair point.

What we do thrives on exceptions. I wouldn’t be able to think of a good example, but by chance, I read of one in the NYT yesterday:

When asked what market research went into the iPad, Mr. Jobs replied: “None. It’s not the consumers’ job to know what they want.”

Do what works.

Are there psychological terms for this?

So you’ve decided to strike out on your own, by freelancing or start a company. The first day you get up, you realize you’re free. No office, no working hours, no one staring over your shoulder, no constant pressure to have your butt in a chair doing nothing. Great, right?

Well, kind of. The first month works great: you’re super productive and everything gets done in two hours, and then you go  home and watch TV the rest of the day. Then your subconscious realizes that if you stay home, you can do nothing but watch TV and surf the web all day, right away.

So you go to coffee shops and libraries. For free, or the price of a latte, you can get wireless, and work with fewer distractions. [1]

There’s only one other big problem: the alarm clock.

[1] If that’s too much angling for seats for you, you can also find a good coworking space for a few hundred bucks a month.

The alarm clock

Credit

The alarm clock is, biologically speaking, the enemy. Studies almost universally seem to show that having less sleep is the equivalent of taking the stupid knob on your brain and turning it up, all day. Any machine that interrupts your natural sleep cycles can’t be good; in nature, it’s essentially the equivalent of having a (mostly) friendly bear wander by your hut at 6 am every morning; you wake up groggy, stressed out, forced into action. An entire company was started and funded around avoiding it.

So, naturally, the first thing you do, once you’re free, is turn it off. The only problem: You’re a night owl. So now you have random hours. Every day you wake up, and you’re not sure what time it’ll be.
Do you just head to the same cafe, whether it’s open for 12 hours or 4?

Cognitive Load

One of the hidden advantages of regular office hours in a regular office is reduced cognitive load. If you have to work every day, but you’re not sure where or when, you spend time thinking and processing: “Is this place open? Is it worth it to commute 30 minutes if they’re going to close at 5? Maybe I should just stay home today. I wonder what’s on my email.”

Continued for too long, this thinking drains your reserve of mental energy, before you’ve gotten to work.

With a rigid, defined procedure, none of this thinking takes place: You roll out of bed, put on your clothes, do whatever and in 30 minutes you’re in the office. And you haven’t had to make any decisions yet. Your mental energy is saved for when you get in and tackle the first work-related problem.

That’s the benefit of reduced cognitive load.

Morning Routine 2.0

Does this mean that you, the freewheeling self-employed freelancer, need to don a suit every single day, set your alarm clock for 4 in the morning, and go for a run of 1.8 miles before coming in for a coffee and commute to the office?

Fortunately, it doesn’t.

The most important thing that you need to have in the morning is not super rigid rules, but a set of algorithms that reduce cognitive load.

Most likely, in the morning, you’re making the same set of decisions, over and over again:

• What should I wear?
• Where should I work?
• What am I working on today?
• How long should I work?

For a full-time employee, these sets of questions have the same answer, every single day. Regardless of whether you’re feeling great or tired, regardless of what you have to do that day or what appointments you have, you have to answer these questions with the same location and the same time.

Using the same answer every day may be easy, but it doesn’t always fit. Sometimes you’re taking calls and networking all day; sometimes there’s a tropical hurricane heading past your state and it’s pouring buckets for the next week.

So, instead of a morning routine, have a morning algorithm. Here’s one I’m starting to use:

1. Pick out clothes the day before, based on weather and schedule.
2. If I have to take calls that day: pull out the laptop and work from home (It’s hard to take calls in a public space).
3. If there’s a meeting/event to go to: find a place to work near the meeting place
4. If it’s before X o’clock: Work at the faraway office (most productive for long blocks of time)
5. If it’s after X o’clock: Work at a nearby coffee shop.

With whatever algorithm you use, the instructions should be so simple, straightforward and brain dead that you can apply them in 5 seconds without questioning them. And once you’ve polished your algorithm over a few weeks of use, you should always follow it to the letter. Amend it with the greatest caution, because once you start changing it every morning, the benefits of reduced cognitive load disappear.

By doing all your thinking beforehand, you can save your brain cells for when your routine is over and your real work begins.

Ever hear the phrase “write your congressman?” Those sorts of actions I like to call “participating in the tide.” Combined, perhaps the millions of people who write with you will together make a difference, but not you specifically.

I would argue that, even for great inventions, most of the time we are participating in the tide, not shaping it. Look at, for example, Alexander Graham Bell. You might argue, “Of course he changed the world! Dude invented the telephone,” and by some definition, you’d be right. But when it comes down to it, the telephone was bound to be invented; there was even someone else inventing it at around the same time, and people argue over who was first.

Ferdinand Verbiest, inventor of perhaps the earliest self-propelled vehicle

William Murdoch, built a working steam carriage in 1784

Oliver Evans, first to get a US patent of a working steam carriage

If actions are individually insignificant, why should we as individuals care what we do?

1. Don’t give up on the tide: History is often changed by movements. The fact that our one contribution is small doesn’t change the fact that if all of our small contributions stopped, so would the world. Whether or not we want to do something more deliberate, being a part of the greater trend is important.

2. Point: This should make us feel better: We needn’t worry about changing the world or making a huge difference. Even if we don’t build a successful social networking service, someone will do it. Even if we don’t make solar power, someone else is still probably working on it.
And many of the more obscure, unique inventions that wouldn’t have flowed naturally out of a lab somewhere have become less important, their impact smoothed over by time and trend.

3. Counterpoint: What if it doesn’t: Perhaps we still have an urge to make a unique, individual difference? If so, then what?

1) Stop wanting: There are arguments not to try and shape society in such a non-inevitable manner. Stalin changed Russia, Kim Il Sung changed Korea, and certainly one way to change (short term, though likely not long term) history is to exert power. But history has not conclusively demonstrated that individuals who held this kind of world-shaping, not-likely-to-happen-on-its-own power have really made the world better, despite their best intentions. Perhaps it’s better just to participate in the tide and make sure we’re moving the world in the best way we can.

2) Work on ideas: A more powerful way to influence the world is through thought. Someone may have invented the press and changed the world if Gutenburg didn’t, but would someone have come up with Marxism, capitalism, or freedom of that press? I suspect for certain ideas, someone would have invented the term. But not all. Would we definitely have open source if no one coined it? An interesting question.

3) ???