Which is your favourite crud, which did I miss?

#HumbleBeginings

What circles do you think you have been added to?

tldr:
Google+ introduces a new type of privacy problems which I can only describe as the inverted personal privacy dilemma. It is no longer an issue of “What I choose to share with Google and then what Google shows to others”, the issue is now: “What friends/complete strangers choose to share about me to Google, and my inability to do anything about it”. The responsibility of maintaining my personal privacy has been partially removed from my hands, and placed in the hands of the people of internet.

Thought experiment:
Put 100 privacy aware/concerned people into a room. What would you suppose is the best way to create a global interest graph/taxonomy for this group of individuals? Algorithmically? Manually? First, ask them to fill out very basic personal information about themselves. We will probably have reasonable success. Next we ask them to fill in very specific personal information/categorizations about themselves. How do they react? Probably not as successful. Now ask the same group of people to write down and categorize the people around them, but remind them that their categorizations will remain secret. Each individual’s privacy has been maintained, and collectively we have created a very rich interest graph/taxonomy which is possibly far more accurate than any automated solution.

Doing this algorithmically is hard* [full stop]. The process of taking a very soft analog characteristic, and turning it into a digital signal can be very difficult to get perfectly. Humans are very good at doing this. This problem situation seems very familiar:

Before people-tagging came out, I think most people would have said that the best way to figure out who’s in photos was to have some face-recognition algorithm. But it actually turns out that the best way is to just have people tagged.
-Mark Zuckerberg

Background:
As Google throws it’s hat into the social ring once again, we have all been treated to a very interesting and exciting new alternative on the web to social networking. I’ve read many very interesting opinions on the new product, some heavily praising, some really bashing it. I’ve resisted my natural urge to do an overview of the UI/general UX. I will not be doing a product or feature review. I will however look at some interesting privacy implications some of these new features/concepts this product has introduced.

Google’s famous mission statement: “organize the world’s information and make it universally accessible and useful” has lead to some of the most brilliant and amazing products I have ever used. Using my own mission statements, I see Google Search (organize the web) of course topping the list, their amazing Maps product (organize the physical world), GMail (organize my digital mail), Google Images (organize the web’s images) etc all as fantastic products. It became clear in the late 2000′s and now into 2011, that the social web and social networking was the next necessary territory grab online. Staying inline with Google’s motto, I personally see Google+ as: organize my social network, or perhaps organize the world’s content with social context for me, or maybe even: organize the people in the world.

The Dilemma:
To me, Google may have accidentally introduced a new type of privacy issue, something I call the “inverted personal privacy dilemma”. I call it a dilemma because while I feel like my privacy is safe from others, I am essentially exposing information about other people without their knowledge. And I use the word inverted because to me, it seems opposite to all of the traditional privacy problems that exist on the web.

This is also a dilemma because I personally don’t like seeing myself comparable to an indexable book, nor do I feel like I am a place on a map. I’d hope that there is a human element that lets me be who I want to be, beyond any indexable attributes or properties. There is already a pretty large index of people on the web, it is called Facebook. I use it, and I provide as much information about myself that I feel comfortable with, and this is what it looks like for me:

The information here was provided by me, and is essentially the most amount of information I’ve ever really been comfortable disclosing with people online. It is the sufficient snapshot of my life which made it easiest for my old friends/coworkers to find me if they chose to look. I personally don’t add anything else, although Facebook gives you the option to do so, and many do willingly. I don’t add music, movies, religion, favourite book, etc, although again, many people choose to do. And that is fine.

Circles:
Interestingly enough, in addition to the basic personal information I’ve added, the Google+ circles metaphor allows a very similar view of my networks. My circles are explicitly created/labeled, private views of my social graph. This data certainly says something about me and my network from my perspective:

Right now, I am notified when someone has added me to a circle, and I can choose to ignore it, or maybe add them to one of my circles. Fairly harmless. From the perspective of my Google+ account, this is a visualization of most of what Google knows about me based on what I have explicitly done on the site (circles/bio etc).

I have added my location to my profile, my alma mater and that I have previously lived in Toronto. Additionally, I have created circles for those properties which further re-enforces the data graph that these are true characteristics of the person Ian Chan. I have chosen explicitly to share this subset of information with the web, not too different than the minimal amount of info I have shared on my Facebook account.

Inverting:
Now let’s invert this graph, or rather, lets visualize how other people on the internet have added me to their circles and what the implications are. It’s worth noting, a large portion of people who have added me to their circles are complete strangers:

Now due to the existing privacy constraints, I cannot actually see which circles I have been added to. This satisfies most people’s concerns about the circles metaphor. But it is easy to speculate how people may curate their graph. If one of the purposes of circles is to allow topic based feeds of data, and narrow cast content to these circles, the above graphic is fairly reasonable for me. At the bottom, I display the expected: Toronto, SF, UofT groups which compliment the same circles I have already created. Given the bi-directional (explicit outbound + implicit inbound) taxonomy group agreement, it is a fairly strong/confident signal about my personal characteristics. The more people who add me to their SF circle, likely the stronger that signal becomes. Javascript, UX, startups is another group of circles I can imagine myself being placed in. Since I often talk about these topics, and am active in each respective community, it would make sense for the industry connections I make to slot me into these bins.

The last property, Poker, is something that gets a bit more interesting. Yes, I know a lot of people through my hobby playing poker, and it is not unrealistic for one of my friends/acquaintances to add me to a poker circle. The action of them doing so creates a new edge on my implicit interest graph for which a data point now connects me with the  ”poker” taxonomy entity. The more people who add me to a “Poker” (or similar) circle, the stronger that graph edge becomes in the global taxonomy. Now, I am pretty open about my poker hobby, but I wouldn’t explicitly add that as a hobby on Facebook;  Mostly because I don’t need advertisers knowing that “gambling” is an enjoyable pastime of mine. As I mentioned before, my Facebook interests are basically empty, and as a result, I rarely get ads targeted to me beyond the generic targeting for a “male in SF”.

Implications:
Advertising and control. The prices for targeted, contextual ads are significantly higher than non-targeted ads. Facebook learned this a long time ago, especially in the social context. From the Facebook advertisement documentation:

I like this (Facebook’s) model. It is within my control. Poker is obviously just an example here. I would not be devastated if I were targeted for a poker ad, but I might be a bit uncomfortable with it. But there are a lot of other characteristics about myself that I would not want to share with an ad targeting system. My religion, my political leanings, my ethnicity, etc are just a few characteristics that are all reasonable circles one may innocently add me to. And the current circles model allows essentially anyone to “tag” me with whatever they want, as a result, constructing an interest graph on my behalf.

Advertising:
Google+ does not currently have display ads, which is very nice! However it is clear that their large app suite is pushing towards some kind of convergence, so there will be ample advertisement opportunities down the road. Could information/context provided by another person about me really find its way onto my screen? Lets try:

Unfortunately, most recruiting emails I get are not too far off from this spoof

I sent an email to myself as Joe Co-Founder who was looking for a Rockstar Ruby Developer. I did not choose to get this email, though I did choose to open it. And (as expected), the adverts are all tailored to the context of the email. This is good advertising, this is targeted advertising, and this is only the beginning.

Choice:
Choice is always nice. Currently in my bio, I can choose to omit certain bits of data that I feel are too personal. On Facebook, if I had chosen to not associate myself with my high school in Toronto, I would simply remove it from my personal information. With that one simple choice/action, it would eliminate any explicit way of connecting me to the school I went to before University. On Google+, I am almost certain that people have started to add me to the “Northern Secondary High” circle, because that makes perfect sense, because that is exactly what circles were designed for. I have no choice in the matter, I have been labeled. I can choose to not use Google+ of course, but I enjoy using Google products. I could choose to not use circles at all, but the current UX requires me to put everyone into at least one circle.

Parallels:
Some have described “Circles” as being analogous to Facebook lists, for which I would disagree. While the concept is similar, the privacy model (in the context of this article) is very different. I can only add people who are my friends to lists. People can only be my friend if either they or I have permitted the connection. Therefore, the people that may be adding me to lists are within my accepted/trusted circle. Some have also drawn parallels between circles and Twitter lists. Again there are differences. While the asymmetric nature of the following graph is similar to Google circles (anyone can add me to a list), the difference is that these lists are all public to me.

Lists I have been added to. Lists created by people I know/people I don't know...

I’ve been added to ~100 or so lists, and I can check them out and see what people have identified me as. Nothing too surprising, but the information is all there. If I found a list I did not like being in, I could simply remove myself from said list*.

To me, Circles are not like either Facebook/Twitter lists. I see closer parallels in other places on the web:

delicious.com: tags associated with a specific bookmarked url

stackoverflow.com: Tags associated with a specific question

This “tagging” model is much closer to circles in my opinion than anything else I’ve seen on the web. And as I said before, it feels weird being tagged/indexed like a book, a question, a url, especially when I have no control over it.

“Ian, you are a rambling lunatic…..”
I’ve chosen to *really* nitpick a single feature. I really don’t think there is some super secret master plan here. The purpose of this entire post is to just be a talking point about privacy, the social web and ad targeting in general. I think that Google+ is great. I really find the circle metaphor an interesting approach and concept. I do not believe that they built it as some kind of secret way of mining data about people, and truly hold the opinion that it was designed to empower users. However I acknowledge that creating a rich, strong taxonomy/database of people and characteristics would be incredibly hard to do purely automated. I kind of feel like the cirlces metaphor is the greatest Mechanical Turk task ever created which is organizing and labeling the people of the world. What circles do you think you have been added to? If you decide to add me to a circle, maybe just add me to your “following” circle. And while your at it, you might as well just follow me on Twitter, I’ll probably follow you back….

Organizing the world's information, one circle at a time…—
Ian Chan (@chanian) July 21, 2011

Editor’s note: This is obviously an expression of my own opinions and not my employer. While my employer is publicly being considered a competitor to G+, I would hold the same thoughts and unbiased opinion regardless of where I worked. Also, thanks to @michaelcvet for the feedback and revisions on this post.

[*] On the difficulty of automation: If you disagree, stop reading my blog, make the product and make millions!

[**] On Twitter lists: I acknowledge that there are some complications with private users/lists, but that is far from the average use case.

I’ve been experimenting recently with a bunch of cool photosharing apps on my iPhone including: Path, With, Twitter, Facebook, Denwen, Color, Dailybooth… (takes breath). As much as I enjoy each one in their own unique way, none of them (yet) are able to capture the true feeling or ambiance of a particular situation, though Path is probably the closest. To be fair, most of these apps were not designed to do such. I’ve always found that regular photos (camera phone for instance) can decently capture a moment, but they often come off as rather 1-dimensional (figuratively speaking). So as an experiment, I documented ~72hours of my life by taking panoramic photos along the way. Below are shots of where I found myself wandering to over the last 3 days. For me, these wider photos do justice to some of the lovely views I get to see on a daily basis.

I am not a photographer, these are far from perfect, but I think they do a great job of giving the viewer a sense of where I was and what I experiencing. I recall reading that the switch for TV’s from 4:3 to 16:9 ratio was to make a more natural aspect ratio for the human eye. Well, even at 16:9 (which might approximate my eye’s static view), I often tend to move my head left/right, so hopefully these wide/POV shots are more emmersive. Lastly, I have to tip my hat to the team who made PanoApp, a great $2 iPhone app that does all the stitching work for you. Go download it and post stuff.

Anyway, here is 72 hours exploring in SF. This is not realtime, this is not broad casted/narrow casted, this is not person tagged nor is it geo tagged. Just some simple photos I thought some people might enjoy.

SOMA: The view from my desk at work, I have never used the monitor on the right

SOMA: Every Friday at TwitterHQ, teatime with drinks, live music and serious QA... I'm holding 2 beers.

SOMA: Walking over Howard St on the Yerba Buena bridge. I'm listening to Metric.

Embarcadero: My Thursday poker game with some of the nicest guys in town, though I'm about to lose a big pot

Union Square: The Powell BART station, heading out for food. I'm craving Chinese food

SOMA: Chatting with my coworker @alan about my panorama experiment. The pano is so meta.

Nob Hill: The top of the hill, just a few blocks from my apartment. Choice: Russian Hill ahead, China town the the right

Russian Hill: One of the steepest/funniest hills in the city. I'm thinking about tipping a car over (Broadway/Jones)

Russian Hill: Relaxing at one of my favorite "hidden" spots in the city. A random dog sits down beside me to enjoy the view

Dolores Park: Tons of people enjoying the nice weather. I'm wearing my pink sunglasses to fit in

Mission: The Summit cafe, a new but prototypical cool/geeky tech hangout with great coffee. I'm holding a fancy Cappuccino

The Marina: You can see the Golden Gate to the left, and Alcatraz to the right. It is really windy

Union Square: Tons of tourists lining up for the Cable Car and enjoying some local street performers

Mission: 16th St. BART, a shot of me going home after a long day of wandering. photo by @moizsyed

We see thousands if not millions of moments just like these on a day to day basis. Hopefully I’ll remember to stop and capture a few of them for me to revisit and share in the future. On a closing note:

I think Im developing an unhealthy obsession with taking panoramic photos…—
Ian Chan (@chanian) June 12, 2011

Problem #1: QUnit.done misfiring

QUnit.done = function(qunitReport) {
  console.log('Done!');
};
setTimeout(function() {
  test("helloworld", function() {
    expect( 1 );
    ok(true);
  });
}, 100);

The timeout here simulates the delay that an asynchronous dependency might create when QUnit loads up all the tests. In the above example, the QUnit.done event will get called several times (a similar report was addressed here, but I am still seeing the issue). This can be particularly problematic if you are using a test runner to automate your QUnit test suite, and are waiting on this event to complete your suite.

Problem #2: Missing/Empty test cases

window.setTimeout(function() {
  asyncTest('test1', function() {
    expect(1);
    ok(true);
    start();
  });
  test('test2 - Something has gone horribly wrong!!', function() {
    expect(100);
    console.log("Yes this test run");
    ok(false);
  });
  test('test3', function() {
    expect(1);
    ok(true);
  });
}, 150);

In this example, the second test (or actually any test between the first and last) will be listed, run, but the results of the test will be ignored. This can be particularly dangerous since they will be visually reported as passed (yet empty) tests, and the QUnit.done results object will not show any errors. In this situation, it seems like QUnit collects the list of tests, but during the execution of the individual tests, something goes wrong.

Note that the 2nd test has 0 reported asserts, even though the test itself was actually run.

Timeouts (independent of actual asyncTests)/asynchronous behavior tends to throw the test runner off. Since I was unit testing async modules and code that depended on these modules, this was a problem. What I needed was a way to ensure that the test suite was only run after every test was registered, and I could only register each test when I knew the dependencies they relied on were loaded themselves.

Make each test a managed dependency:

QUnit.config.autostart = false;
using('tests/suite1', function() {
    start();
});

For starters, we tell QUnit not to autostart. This will allow us to explicitly start the runner when we are certain the test environment is ready. We next re-write, then load in our test suite(s) as loadrunner.js dependencies, and once that is complete we are safe to begin. loadrunner.js will deal with all the sub-dependencies that may be required in “tests/test1″, and will also block until that test has explicitly reported that it is ready.

provide('test/suite1', function(exports) {
  asyncTest('test1', function() {
    expect(1);
    ok(true);
    start();
  });
  test('test2 - Something has gone horribly wrong!!', function() {
    expect(100);
    console.log("Yes this test run");
    ok(false);
  });
  test('test3', function() {
    expect(1);
    ok(true);
  });
  exports();
});

And when this test suite is run, we see the expected failing test, and also a single done event fired. The screenshot below shows us the desired output, a big glaring warning telling us that something has gone wrong in our code!

I realize now that my choice of example makes this look bad, but we *want* to see this failing test. In this example I also brought back the QUnit.done, which now works.. a nice side effect.

I happened to use this technique (and solution) in loadrunner.js, because I was developing and testing loadrunner modules. You can find a similar solution if you are working with require.js here from the QUnit boards. I’ve put some additional code up on Github here, which runs through a few various test scenarios and how this pattern can be used to create relatively clean/readable test files. I’ve also included a test template for “async-safe” QUnit suites.

I personally felt that this syntax also reads much nicer once you start including 10′s if not hundreds of test files. loadrunner will also load regular javascript files in the same fashion so you can mix (if you know for certain that there will be no sync probs). I found that by treating each test file as a managed dependency, you get better testing isolation, less potential for test poison/side effects, and more control over your test context with very little footprint. Lastly, it allows us to avoid making everything an async test just because a specific test happens to load an async dependency (especially when we are unit testing functionality, not the ability to load things asynchronously).

Hope this helps out someone down the line! Test safely my friends!

I am always interested in looking at large sets of data. Earlier today, LinkedIn released InMaps which provide a simple tool for visualizing one’s network on LinkedIn.  After a few seconds of processing, I was presented with this colorful and interactive visualization of the ~200 connections I have on LinkedIn. Now I don’t have quite as many connections as others, but I was still very impressed with the powerful bits of information I was able to quickly get out of this graph. The plotting algorithm did a pretty good job, as I see 3 major clusters which are composed of my University of Toronto connections, the connections I made at my previous startup Thoora, and the connections I have made (mostly coworkers) since I moved to San Francisco for Twitter.

The legend for the clusters was manually added, and I am curious if LinkedIn is using that data for something other than just a label. Either way, a very impressive, free, quick view of what I can imagine must be a ton of data. I can only predict over time that orange ball will grow, and likely split into two clusters as I connect with more coworkers, and connect with more tech geeks in the city.

This tool and graph gives some insight into the power of the data and network information that LinkedIn is housing. I constantly tell my friends and coworkers about how there are many *hidden* values of LinkedIn for which I appreciate, this tool being one of them. Unlike my Facebook account, I am not terribly interested in the general day-to-day updates of my network, but much more interested in the network itself. Who do I know? who am I connect to? and who might find me because I am connected to someone else? I’d say that to me, Twitter is all about the sum/collection of the individual tweets. Facebook is about the sum of the individual micro-interactions, and LinkedIn is all about the strength, and the quality of the social graph itself. This tool helps reinforce my opinion that my LinkedIn graph is much stronger and stable than my ever growing graph on Facebook.

Some nice slick looking vector graphs backed by Rapheal.js and smooth as butter zooming and panning powered by Microsoft’s Seadragon Ajax. This reminds me that I need to: (a) Find a cool large public data set. (b) Visualize some data. Try it out for yourself here.

I recently received a Wacom tablet+stylus as a gift and could think of no one else to draw for my first sketch but Alan Turing. This is a 4 layer Photoshop trace atop a famous photo of one of my favorite historical scientists. I figured it could be a late tribute to one of the guys who started it all. On this day (November 28) in 1942, Alan Turing was sending a report from Washington, D.C. describing his 2 week tour of America and their code breaking efforts for WW2. I’ve always appreciated his subtle humor and lingering disappointment in his writing/dialogues, this report is no exception. Always up for a challenge, and (seemingly) rarely impressed or satisfied; breaking German encryption codes and theorizing the modern computer wasn’t bad for a 30 year old math geek from West London.

“I’d say we have come a long way since Zygalski sheets and Hut 8 in Blechley Park, Alan.  Don’t forget to thank Welchman for that diagonal board.  R.I.P.”

Above is an overlay of every (CrunchBase listed) startup in San Francisco.  In an afternoon hack, me, @alan and @thetylerhayes scrapped the crunchbase API then mashed it into Google Maps to get an interesting view of the city.  I am always amazed at the amount of startup culture in this city (hence I moved here), but never had a chance to really see it from this perspective.  My office and apartment are somewhere buried underneath one of those pins.  I’ll try to put up a more interactive version of this chart sooner than later.

Who do you bring to the fight when facing a tough market, timelines and competition?

While excellent engineering talent is a must, there are many overlooked or under-appreciated roles that I believe are essential to the success of any software company – at any stage.   I often hear the excuse:  “We are too small of a company to hire a ________”.  Well guess what?  Your competitors have one, or maybe more, so SOMEONE at your company has to be taking on these responsibilities.  You might be lucky enough to have these roles filled, perhaps these responsibilities are shared within your company, but ignoring them will soon land you in a lot of trouble.  Lets get started with the 3 P’s that I believe are so important for any software project to succeed.

Product Manager:
“I make sure we build the right product for our users and company…”
Alter Egos: Product Owner, CEO, Business Analyst
Answers the “WHY?”

Don’t need me? Then….

• Who is going to provide business/market validation for large product and business decisions?
• Who is making sure that what we build has a realistic product market fit?
• Who comes up with ideas that facilitate the success of our business model through useful features?
• Who mandates creating new, removing old, updating existing features?
• Who is making sure our product strategy has a vision, time-line and plan?
• Who is going to create a spec/scope, so we don’t feature creep our way 6 months past our deadline?

Without me….
The end product will only be valuable to a small slice of a niche market that doesn’t actually exist.

Product Designer:
“I make sure the product is actually good, looks great, and works well…”
Alter Egos: UX Designer, UI Designer, Interaction Designer, Visual Designer
Answers the “WHAT?”

Don’t need me? Then….

• Who thinks about how users are actually going to use our features and what those features will look like?
• Who is going to make/take wireframes and turn them into something someone can actually use?
• Who is going to design a product that is visually/functionally comparable to the industry standards/competitors?
• Who is going to tell the Product Manager that 9 banner ads are 9 too many?

Without me…
The end product will be an awkward interactive wireframe using bizarre UI elements soon to be seen in the “before” section of Steve Krugg‘s next book.

Project Manager:
“I make sure that the project we are engineering actually gets finished…”
Alter Egos: Scrum Master, Project Lead, Team Lead, Project Coordinator
Answers the “HOW/WHEN?”

Don’t need me? Then….

• Who is making project estimations, to give realistic timelines for launch?
• Who is protecting the time of the engineers, keeping the development train moving and free from endless planning meetings?
• Who is overseeing the overall progress of the engineering efforts?
• Who is ensuring that the 2 months planned for testing hasn’t been shrunk to 1 week due to unrealistic deadlines?
• Who keeps engineering efforts focused and on task when inevitable distractions arise?

Without me…
The product will be released 1/4 year late, with 1/9 the features promised, and 1/16 code tests coverage….

(And with that kind of terrible geometric progression, you’re looking at only getting PI^2/6 of your project done!)

Building Something Great
Engineering is so important.  Engineering is usually responsible for answering the “HOW”.  But this is an engineer writing about and appreciating all that are (normally) outside of my call of duty.   Summing it up: there are a lot of very important duties that are outside the realm of traditional engineering responsibilities.  These are 3 very important roles I have come across, and duties that I feel should never be overlooked, no matter how big or small the company.  Looking back at that photo of the chessboard I choose, I’d hate to imply that developers are the pawns, but the point was to remind everyone that there are a lot of difference piece on the table.

(Note: Searched for an hour for proper photo credits of this great shot with no luck. Let me know if it’s yours or need to go down)

Screenshot from movie trailer: And yes, I realize there is now a fake FB account this *character* in the movie...

Spoiler alert, I don’t really talk too much about the actual film’s “plot”, but if you haven’t seen it, watch the trailer here – watch the movie in mid September, and then come back.  I am not going to reveal the *twist* ending, go pay and watch the film. I was very fortunate today to be invited to a screening of the new movie Catfish.  It is a new documentary/reality-thriller about an online relationship that goes sour.  I knew very little about the film other than a very intriguing trailer that was floating around YouTube.  The film was quite enjoyable, and it did prove to be the emotional roller-coaster as promised, however there were a lot of mysteries behind the nature of the film itself.  In this post, I wanted to outline what I found fascinating about the  film, what I found unsettling about the film, and address the biggest question surrounding the movie:  Were the events in this film was REAL or FAKE?

Well… it was all real..
To be quite honest, I assumed it was a horror/mockumentary.  It was marketed as a horror/mockumentary…. When I saw the trailer I assumed that, when I watched the film I began to question that, and when I sat through a live Q/A session with the creators, I was really really confused.  There have already been other posts which outline the questionable nature of the film itself, and even at Sundance, an audience member apparently asked whether it was real or not (to the surprise of the audience and the film-makers).  I am generally skeptical about everything I watch, I am from the internet generation where we are trained to question almost everything we see.  After all, without questioning the genuine nature of the film would be horribly ironic and would miss the whole point of the movie itself.  That said, I don’t feel bad for asking the question in the first place.

Not surprisingly, opinions online range from “C’mon, it is obviously fake”, to “No, seriously, this is actually real.”  These varying opinions to me point out that it is not obvious.  At a high level it seems so outrageous that it cannot be true, however, for now I’ve come to the uncomfortable conclusion that the movie is real.  Here is why:

1. A live interview with the crew: After the film, there was a great QA session with the filmmakers who took many questions from the audience.  Almost every question was about the circumstances of the events, and the interactions with the real people in the film.  I can’t imagine the act needs to be drawn out that long…  They seemed like genuinely nice guys. *(update, I just met them, and all 3 of them were awesome guys…)

2. An almost overwhelming amount online evidence leftover from the events in the film: After Googling around for literally like 10 minutes out of curiosity, I found enough evidence to convince myself within reasonable doubt that it is real.  I am not going to post links, but you can easily find tons of links to the people and events referenced in the movie.  It took me about 2 minutes to find the actual Google Street View location we see in the film of the house, which made me very uncomfortable.  At that point, my personal creep meter was at a limit and I backed off.  I had seen enough, and want to respect these personal boundaries (which I had already over stepped)

Then why did I come to such an uncomfortable default conclusion?
It was marketed as fiction, so I was compelled to do some research.  I really didn’t dig that deep.  After watching the movie, and hearing about the “great viral marketing”, I jumped online.  Being a huge fan of LOST’s famous alternate reality campaign, and probably being one of the few people who saw all the extra content LOST scattered online, I figured it was all part of the experience.  But then I guess I dug too deep. I am presented with the option of either accepting that the story is real, or that it is fake and some very distasteful viral-marketing department has made some questionable marketing decisions. Given the evidence presented to me, I don’t want to look any deeper.  If everything I unearthed is an elaborate alternate reality ploy by a devious marketing department, then I will have been duped.  But I really don’t want that to be the case, and will just default to the assumption that I saw an interesting accounting of some pretty bizarre events miraculously caught on tape by the film crew it just happened to involve.

So then the biggest mystery to me now is this: Why did the marketing team (presumably Rouge Media/Universal) make it so ambiguous of whether the film is real or not?

• The biggest draw of this film, which intensified the emotions exponentially, was that it was all REAL, why hide that?
• If the actual film makers weren’t there for the 1 hour QA after, I’m fairly certain that 1/2 people in the audience would have assumed it was a work of fiction.
• There was very little mentioned in the film that really emphasized that it was all true events (no more than say, Blair Witch).  Again, why down play this?
• Why did they market this movie like a Blair Witch / Paranormal Activity knock off if they were legit?
• Why was the trailer’s copy-writing so confusing?:  “Not based on a true story, not inspired by real events… Just True”. Yeah I get what they are saying, but that is just utterly horrible copy!
• Why include real names/addresses in the film (and trailer) that easily link back to online profiles of the real people in the film?
• Why has no one else (seemingly) lifted up a few rocks before questioning, discovering the same (in plain sight) evidence I found?

Why not make it PAINFULLY clear to the viewers that this is a true story?  That way you don’t get speculations like this, or this, or this, or this….. I get that it creates buzz, and press, but it takes away from the story (in my opinion).

I guess, it is not that important to know if it was real or not, but I fear the majority of the American audiences will assume it is fiction, and that is unfortunate.  Either way, it was a highly enjoyable film and an incredibly story.  Perhaps the ambiguity will minimize the public attention/scrutiny of the people featured in the movie.  Perhaps the success of this movie will help bury the references to these real people, and hopefully protect them from the Internets.  I’ve purposely left out any names, links, or references to avoid the robots and spiders out of (what I feel is) courtesy to the people of the film.  I REALLY hope that people respect the boundaries of privacy of these people and that everything works out well for the filmmakers and the family.

So what are you thoughts?  End of the day, maybe we just saw the extended version of the HOPA girl…..

Pretty bizarre tutorial no?  But after going through this experience (just shy of 2 months ago) I realized that there are a lot of things I wish I had done / known before.  A recent visit from a friend from Toronto had me thinking that others may benefit from a post like this.  This info is specific to San Francisco, however I am sure it applies to many other cities.  I have to thank @bentlegen and @shazow for the advice I got while moving in, I intend to pass the baton.  For a great post on how to decide where to work (if you are fortunate enough to have options), checkout @shazow’s post on: A check list while considering offers.  This post outlines all sorts of issues I ran into, feel free to skip ahead to the parts you might find useful.  I will cover things like: getting a visa, finding a place to live, and how to efficiently spend your first week in the city.  As a disclaimer for all the immigration advice, this is just through my experience.  I am not an immigration lawyer and if you have specific questions, I suggest you consult one.

Preparing for your work VISA
[You just got your job offer]
Congrats!  Hopefully the job you got is sponsoring you for a VISA, otherwise you can stop reading this post and start looking for another job.  The company may or may not help you actually get the VISA (ie immigration lawyers etc).   I applied for a TN1 VISA, you may be applying to that or something similar.  You will need:

• Your original University Diploma (out of frame).  Yes this is a pain in the ass, but that over-sized piece of paper is actually the only proof you graduated.
• An official copy of your transcript.  For CS grads, this is the only proof you actually took computer science.
• Birth certificate + (drivers license) + passport, to prove you are Canadian
• An offer letter with a start date, to prove you have a job (provided by the company)
• A specific summary of your job duties, to prove you are an Engineer (provided by the company)
• Information fact sheet about the company, to prove it is real (provided by the company)
• Signed and filled out VISA sponsorship paperwork (provided by the company)
• (And if your company’s lawyer is really sharp) A copy of an old letter circa 1996 from an immigration judge confirming Software Engineer == Engineer

I’d suggest getting the official paperwork from your University as soon as possible, because you don’t want to be waiting on them right before you move!

Finding a temporary/permanent place to live
[You committed to move, and now need a place to live]
As per the excellent advice from @shazow, I needed to spend some time in the city to get an idea of where I wanted to live.  Signing a 1 year lease then later finding out you live in cracktown would suck.  There are a lot of temp housing options (perhaps offered by your company).  Might I suggest using AirBNB?  I used it, and it was fantastic. When it came to finding a permanent place, I used craigslist, saw like 15 apartments and it worked out well.  Take note that since you have no credit history and no SSN, you might have a hard time.  The best solution is carrying a copy of your offer letter + your salary agreement to your house visits.  You may also have to pay an inflated down payment, which is reasonable.  You will likely have to create an account with PG&E, to pay for your heat and electricity, but this was probably the most painless process of it all.

Have a reasonable amount of liquid cash available in America
Since your Social Security Number won’t arrive for weeks, don’t expect to get paid for at least a month.  If you move into an apartment, you will need a down payment and first months rent (and maybe more, since you have no credit).  You need to have a plan on how you are going to move your funds from your Canadian account to your American account.  Your landlord will NOT take a personal Canadian cheque.  Your bank will not cash a personal cheque, and if they do, you will need to wait weeks if not longer for it to clear.  Make sure you have a way to get your money to an American account as soon as possible.  I did a wire from my Canadian account to my American account.  So before you leave Canada, ensure that you have ALL the wire info needed and that they will do the transaction without you in the physical branch (most banks do not).  If that isn’t an option, transfer funds to your siblings/parents account before you leave.  That way they can do the wire for you once your American bank account has been created.

Applying for the VISA itself
[All your paperwork is ready, and you are ready to make the move]
I got my VISA and paperwork at the airport YYZ.  I arrived 3 hours before my flight which gave me ample time.  I went through regular immigration, and told them I was applying for a TN1 work VISA.  They directed me to the additional screening room where I was interviewed for about 20 minutes.  You’ll need to pay about $50 USD for the VISA itself, they take cash, so have that handy.  Many friends have suggested driving to the border before hand and getting the paperwork done there.  That isn’t a bad idea.  I was pretty certain I had all my papers in place, and chose to just get it right before my flight, your call.

Convincing yourself (and US immigration) that you are an Engineer
[You are walking into the US immigration office]
Note:  you are probably applying as a NAFTA Engineer, however some enter as a Computer Systems Analyst, this section still applies.  Forget about all that time we spent in College bashing engineers, US immigration has a short white list of professions that they let through the border.  You are now an engineer.  To all the “actual” engineers reading this (and I don’t mean P.Eng I mean people who studied engineering) you still need to convince the immigrations officer you are NOT a code monkey.  You will need 2 main things to get past this part.  Your transcript should make it clear that what you studied is related to your responsibilities.  On mine, even though I studied Computer Science, my transcript read – Computer Science: Software Engineering Specialist.  I also took a course labeled CSC301: Intro to software engineering.  Seems silly, but the officer glanced through my transcript, saw the “software engineer” keyword and moved on.   Hopefully your new employer sent you a list of job responsibilities.  If they know what they are doing, you will notice that “writing code” or “programming” is not really highlighted, even if that is what you think you are going to be doing.  Memorize this script, and job duties because they will ask you.  Don’t ad lib.  The second you say something like: “I don’t know, I guess I’ll be writing code or programming stuff”  the CODE MONKEY alarms go off and a net will drop down from above you and they will take you away.  Maybe you aren’t a code monkey, but to reiterate almost every NAFTA/Immigration help website I’ve seen:

“COMPUTER PROGRAMMERS do NOT qualify for a TN-1 visa”.

Stick to the script (hopefully provided to you).  Assume the officer you are talking to doesn’t know anything about computer science / computer engineering, but also assume that they are trained to flag certain keywords like: programming, coding.  Your official job duties should include things like: “Using mathematical models to predict and analyze performance and stability”, and things like: “Applying concepts in theoretical computer science to improve and enhance existing software systems.”  Which is all technically true.  If you are working for a larger company, they have the word-smithing down to a science.  If it is a smaller company, and you are the first Canadian employee, then maybe not.  If you see something like: “Writing frontend code in Ruby/HTML to add new features to the website.” don’t be afraid to confirm with HR/your contact that it is OK.  Don’t confuse this with the original job post you applied for, your official job responsibilities will likely be more (intentionally) esoteric when you get your official job offer.

Lastly, don’t lie.  “So… will you be writing code?”.  “Yes, I will likely write some code … to predict and analyze the performance and stability of the system using mathematical models..”.   It’s like when your girlfriend asks you “Who is that girl you were with?” And you say, “That wasn’t a girl, that was a man with long hair….”.  If you say no, or lie about anything, they will catch you in the lie and turn you down.  It happened to the gentlemen in front of me, it was quite unsettling.

Get your Social Security Number
[You made it into the country, now what?]
Without this, you wont be able to get credit, get paid, get healthcare.  Get this as soon as possible.  I personally ran into A LOT of problems here so hopefully I can offer some advice.  The office suggests that you wait 10 days after entering the country before you apply, I did not and got screwed.  Here is how it works:  You apply for a VISA at the border, they give it to you and staple it to your passport.  They then (at some point in time) manually enter your VISA information into a computer.  That record (at some point) appears in the system for the SSN offices to see.  These offices are disjoint, and the 10 day wait is seemingly a “it should be done by now” buffer.  Since I went early, it took them about 3 weeks to realize that my papers hadn’t been processed at the time of my original application.  Hopefully the same doesn’t happen to you, but since they give you no feedback, I only found out when I went back in to see what the hold up was.  Print off the SSN application form located here, perhaps leave the mailing address as your office, since you probably don’t have a place to live yet.  Find your local SS office, in San Francisco there is one at 7th and Mission, which is utterly terrifying but conveniently close to downtown.  Bring in that form with your passport, VISA (which should be attached to your passport) and birth certificate / driver’s license.  Wait as long as 10 days after you enter the country for the reason above, I wish there was a hard number but it seems really soft, and I got screwed.  After you drop off the paperwork, if all goes well, your SSN will be mailed to you within 2-3 weeks.  I HIGHLY recommend going in again after 1-2 weeks just to check the status, they might give you the number on the spot, or they might tell you that there was a problem they forgot to tell you about.

Open a Bank Account
[You are tired of incurring heavy transaction fees every time you need money]
May I suggest opening a bank account as soon as possible?  As early as the day you arrive in America?  Bank of America was seemingly the only bank that allowed me to create an account without a Social Security Number.  So do that, it takes like 20 minutes and they give you a credit card, which is mandatory for a phone/internet/cable.  Lastly, once this is done you can transfer over all your Canadian money and start writing cheques from your American account.

Get an American phone plan/number
[You realize that no of the real-estate people will email you back]
Guess what? You have no credit or SSN! So be prepared to pay a $500 deposit if you choose to go with AT&T.  This is not negotiable.  Having a call back local phone number is pretty damned important as your try to find a nice place to live.  Do this as soon as you can.

Welcome to the United States of America
[You are happily settled into your new job and home]
I hope you found some part of this post useful.  If you made it through the border safely and are living up in San Francisco, please give me a shout and I’ll buy you a drink and be your one man welcoming party.  This city needs more Canadians so we can setup the very first Beaver tail shop in California.