export your Sidekick contacts

One of the various flaws of the Danger Sidekick service--besides its lack of database backups--was its lack of interaction with the outside world. To my memory, like so many things, this was T-Mobile's fault, but however it happened, exporting your data required buying a third-party sync application, which didn't work as well as one might hope, and was eventually not an option for Mac OS X. It's also pretty lame to have to pay $25 to get your data off the service because you plan to stop using it.

In the run-up to Chile, I explored ways of getting my address book off the backend service, and discovered that through a combination of functionality (likely not on purpose), you can actually export your contacts.

1. Go to the Mail section of the Desktop Interface.
2. Start a new message to yourself.
3. Click "Attach vCard." This brings you to the Contacts screen, where it will show your "A" contacts.
4. Under the "Attach Checked vCards" button, there's a "View" selection. Select "All."
5. Click the checkbox column header to select all your contacts.
6. Click "Attach Checked vCards." This brings you back to your email in progress.
7. Send the email to yourself (give it some time to send).

Some bullet points about the result:

• The email will have one vCard for every contact.
• Assuming you don't want to import them by hand into anything, GMail will let you download them as a .zip file.
• I then merged them (using cat in the command-line shell) into one giant .vcf file.
• The only thing on Linux or OS X I found that will sensibly parse a multi-contact vCard is Google Contacts.
• Google can then export the list in Outlook or other formats.
• I only exported 130 contacts. I don't know what happens if you have the limit of 2,000. Should be fine. Will definitely be slow.
• Plan to do some hand-editing. There's all kinds of crazy "CLASS: PUBLIC" notes and weird wrong birthdays and stuff in the vCards.

Enjoy!

hidden history

In software, we learn that many things that now look like horrible choices actually started as the reasonable solution to a problem. I don't want to give too much benefit of the doubt here: most programmers are idiots, and a lot of the time it's actually a bad choice. Those of us who do horrible things to software on purpose salve our wounded consciences by adding a useful comment with a bug number, so future generations can look up the story and not hate us. Then we drink until the pain goes away.

One of my favorite code stories is the Hardest Bug Of My Career, which of course happened at Danger. It took me about 2 months to figure out, spending 2-3 days a week on it. For those of you not in the trade, that's an absurd amount of time. Most bugs yield their secrets much more quickly, but this was important. A critical step in our Sidekick-data service communication, called the "Dropbox," would periodically fail with an out-of-memory error as my server tried to store it in the database. Fair enough, it was a 3MB XML document...but the server had 2GB of memory. Finally I traced it to how the XML parser would accumulate a block of text, which used a totally simple and reasonable approach, which given a 3MB block of text would use up all the memory and bring the machine to a screeching halt. The problem was with blocks of text in what's called Base64, which are defined to have newlines every 72 characters; in order to fix the problem, I had to remove those newlines, which is a crazy, screwy-looking thing to do. And then a few weeks later that broke our email delivery, and the email guy had to restore those newlines, which is also a crazy, screwy-looking thing to do. We commented our changes, a courtesy not often extended by our predecessors.

On a vaster scale, you can read Joel Spolsky's excellent classic, Why are the Microsoft Office file formats so complicated?.

I was reminded of all this because the professional soccer team, based in Valparaíso, in a Spanish-speaking country, is the "Santiago Wanderers." This makes absolutely no sense to me. The supposed history in Wikipedia tells me:

Despite its place of origin, the founders decided to name it Santiago to distinguish the club from another team in the city that existed at that time, the Valparaíso Wanderers. On several occasions, attempts were made to replace Santiago with Valparaíso as a way to ratify the link with the city. However, the idea has been highly rejected by the fans.

Okay, fair enough. Does that actually help? Now I want to know:

1. Why didn't they call it "Puerto Wanderers" or something else still related to Valparaíso?
2. Why is it "Wanderers"? In English? For multiple teams?

There's always just more questions.

back in the old days

At Danger, we had a team of about thirteen people, each responsible for one or two different servers. All the servers running together is called a "service", which provides all the functionality for the Sidekick: data storage, instant messaging, web browsing, email, etc. Keeping all of these things working in sync is, as we geeks say, "nontrivial." Bugs internal to each server--say my data server is crashing if it gets too much data--are compounded by bugs in how the servers interact: I fixed my data server problem, but now it outputs data slightly differently and the mail server can't handle it. It starts out looking like a mail server bug, but in reality it's a bug in the communication between the two servers. The sane way to manage this problem is to put your code through constant functionality and interoperability testing, releasing it as often as you can (iteration) and always putting all the parts together (continuous integration). So every day we "rolled" the previous day's code out to a developer service that all the engineers and most of the other tech staff used, called "Daily". Because we used it, we usually noticed immediately if it was broken and we were responsible for fixing it. One rainy day in my first year, I meandered into the office at my usual early hour, about 8 A.M., and ran into another Chris, the head of IT, who informed me that the network was down on the engineering floor, due to flooding in the network closet.

This was a three-story building.
"We're having a flood on the second floor?"
"Yeah, leaking through the HVAC vents. Water was cascading down the equipment rack, it was great. There's a tarp over it now, we'll see if it dries out okay."
"A tarp."
"Yep."

Only TJ was in the office. TJ is, to put it mildly, a diligent worker, and yet there he was in the corridor, not working.

"Hey, TJ." "Daily's down. I would have sent email, but, well, Daily's down."

We investigated. Daily shouldn't have been down, because (we thought) Daily's machines were in the nice, dry server room on the first floor. We'd worked really hard to make Daily be reliable in that way. And it was mostly true. The team had a double cubicle partitioned off, with a couch and a wheeled shelf/drawer converted to a liquor cabinet, and a coffee table of sorts, with legs made of two standard beige-box computers, x86-a and x86-b. In the old office, both had been damaged by hallway soccer years earlier, and we assumed they were both turned off, though we couldn't actually tell because the power lights were broken and we were too lazy to check the fans in back. As it happened, x86-a had a shared drive, containing...the Oracle database libraries. Which were needed by the data server. Which was running in the first-floor server room. Which couldn't talk to x86-a, which was up on the second floor, with its flooded network closet.

TJ and I got out a deck of cards and played Hearts for an hour or so until things were fixed.

Computers are hard.

when to take advice

I only had one class today, since Marcela kept the second class to work on an acto they're responsible for in the coming weeks, so after my one class finished their test, I took off wandering. I haven't spent much time in the far west/south end of town, since it's harder to get to and I've been under the impression there's not much there.

My impression was correct. I traipsed around the Old Port area for a while and except for seafood restaurants, there's no reason to go there. And travelers are universally warned to avoid the area after dusk, usually more because of sailor bar-fights than anything else, though since it attracts tourists for some reason, there's a lot of petty theft there as well.

I started wandering uphill from the Iglesia de La Matriz, because the area behind it looked intriguingly decrepit. I figured it wouldn't be nice or anything, but that at midday I'd be fine.

I walked past a kind-looking woman selling candy, and when I'd passed her and was about to enter the next street uphill, she called me over.

"Foreigner?"
"Yep."

"Where are you going?" 

"Dunno, just walking. Thought I'd go up there." 

She shook her head. "Alone? No."

"No? Bad?"
"Always wear your backpack in front, and nothing in your pockets." 

[I've never brought myself to do this.]
"It's dangerous up there?"
"It's complicated. There was another young foreigner, who got beat up."
"Like, flaite? Or something else?" [Flaite is a rich word that covers all kinds of petty criminality and poor taste.]
"Complicated."

"But I shouldn't go there." 

"Definitely not. They'll rob you, and maybe beat you up."

She wasn't very urgent about it, but she was very earnest and well-meaning. It wasn't a hard decision to go a different direction: Chileans are notorious for telling you everything is dangerous, but the universal warnings about Valparaíso don't come from nowhere. And I'm not the guy who's determined to go down that dark alley no matter what new information comes in.

"What's that, you say? That sketchy-looking neighborhood is actually sketchy? Well, I mean, I'm almost there anyway, I might as well keep going. Thanks, though."

I went and had coffee at my favorite cafe instead. Much easier.