To fast prototype and create brand new music apps (web, mobile or physical) in just 24hrs.
To bring together the music industry and the developer community.
To highlight and showcase the platforms and API’s of companies working in and around music tech.
To foster cross-platform and cross-device innovation.
Looks like a great event for local startups and developers to get access to APIs and hopefully distribution.
Music Hack Day Toronto will be held on August 10th-11th, 2013 at the The Glass Factory, 99 Sudbury St.
If you are interested in participating in the fast prototyping and creation of brand new and innovative music apps, be sure to register (tickets are free) for Music Hack Day Toronto today.
Did you know that accelerators are heading for a shake out? We’ve talked a lot incubators, accelerators and cyclotrons. And the proliferation of the accelerator model is generally positive, it started me thinking about a possibility for slightly different model. One that Kevin Swan Follow @kevin_swan posted an insightful comment on the talent shortage for Canadian startups. I don’t think I’m the first to propose this, but it starts to make sense. Incubators/accelerators don’t need to only hasten the formation, creation and ideation of companies. They are fertile grounds to accelerate people. And it’s not just incubators and accelerators, companies participate in HackDays to find talent.
Vuru founders Cameron Howieson and Yoseph West reached out to the Wave Accounting team for advice on building a free, web-based financial services tool. Over time, the two companies traded notes as Wave took on a an informal advisory role, and that led to a sense that Vuru’s talent and direction were something that would be well suited to the Wave Accounting mission. — Darrell Ethrington, Aug 21, 2012 in BetaKit
Vuru was a 2 cofounder team in the FounderFuel (full disclosure: I am mentor in FounderFuel and I now employed by Wave Accounting investor OMERS Ventures). They were building a “investment tracking tools aimed at managing personal finance, which is not something Wave currently offer[ed]”. It was a great fit, a team that had the entrepreneurial culture to make a difference at Wave and a product that filled a known product roadmap gap.
Ok, before Zach Aysan slaps me for being totally incorrect. AlgoAnywhere was not in an incubator or accelerator program. But they had raised a seed round and were building very interesting technology.
The 500px founders met Algo Anywhere at their Pixel Hack Day last year, and were impressed by what the team brought to the table. Algo Anywhere’s tech was originally intended to be sold on an SaaS basis, providing companies with the data crunching power of sophisticated recommendation algorithms, without the need for those to be developed in-house or hosted on a company’s own servers – Darrell Ethrington, July 9, 2012 in BetaKit
The interesting point here isn’t about incubators or accelerators. It’s about founders of early-stage companies looking for relationships and gaps in the market left by other players.
It seems that 500px has been strategically acquiring companies. It looks like both Pulpfingers and Algo Anywhere were part of the PixelHackDay (see photo from TechCrunch). Which gives 500px access to see designers, developers working in their domain space. It’s a great way to round out the product roadmap, Pulpfingers was a iOS discovery application. And they aren’t alone. Hootsuite acquired Seesmic and Swift.
Built to Last versus Built to Flip
I’m not arguing that founders should be looking to build companies to flip. There is lots of conversation about building lasting value. I’m arguing that companies that have raised capital to scale are looking for alternative methods to acquire talent. Get access to the API, build a meaningful service, acquire shared customers and go forward, it’s Biz Dev 2.0 (as Caterina described back in 2006). What’s new to the game for Canada (well Canadian startups) is that for the first time since RIM we are starting to have web startups that are reaching scale and are able to acquire talent, teams and companies. The goal isn’t to look for a acqui-hire or a manquisition, but to look at where working with an existing company or API gives you immediate access to distribution or monetization that you might have to work harder to build on your own.
If I was a developer or looking to get into an incubator program, I’d start looking at the hackathons and APIs that are aligned with my vision where I could accelerate customer adoption.
Find an API (be it local or otherwise) that aligns with your vertical, figure out if you can solve one of your immediate challenges (like distribution and customer acquisition). Maybe strike up a conversation with the product teams at shop. But build something that delights customers and users! Go! Now!
The other day I attended a tech talk hosted by Facebook. Their internal platform team was talking about how they manage the Facebook framework and code base.
The presentation was titled “Code Is Written To Be Read”.
Immediately my gag reflex kicked in. Code is written to be read??? Really??? I literally can’t remember the last time I sat there and thought “hmmm, how readable is this code, I wonder if so and so will be able to understand this”. Having said that, I think I was the only person from a startup in attendance, most were from Google, Zynga, and other larger tech firms. So perhaps I was the wrong audience for this topic.
Whatever their problem is, it is not mine. In my world I have one reason to write code – TO SHIP.
“Code is Written for Users to Use It” (i.e. just ship or shipping is a feature) – that is the startup equivalent mission statement.
And this is where all the “maintainability” coding trolls come in and leave comments like “yeah, but it’ll be huge advantage if we can iterate quickly and get a v2 out and so on and so forth, thus we need code thats easy to maintain”.
No.
Here’s the reality – your product is likely going to fail, so if you wasted time and money making fancy abstractions, doing code golf, and focusing on elite coding craftsmanship… you blew it. You failed. You should have finished it 2-4 weeks earlier instead.
You have to EARN maintenance as a problem. You have to EARN v2. You have to EARN the right to practice expert craftsmanship. If you get there, if you really get to the point where maintaining your code base is a problem for you where many other developers are reading your code… congrats! You’ve succeeded. Go nuts, rewrite everything. You deserve it!! Forget every word I am writing, and go attend the Facebook tech talk and take diligent notes.
But for most of us, we are not going to earn that right. We are going to fail or pivot or leave or get acqui-hired or whatever. That code is going to get thrown in the trash never to be touched again. So how’s that clever FactoryOfTaskFactories abstraction feel now?
And that’s why you probably don’t want to hire Facebook or Google engineers for your startup. And more so, if you are a new grad engineer who aspires to be a startup founder one day, that’s why you don’t want to join Facebook or Google.
Look, it’s not that there is something wrong with those developers. I’m sure working at Facebook or Google is fantastic. It is the closest thing to a tenured prof position you can get in this field. The problem is that they operate under significantly different operating conditions than you do (unlimited money, unlimited time, lots of technical resources, working across massive teams, etc + MASSIVE scale problems, huge performance requirements,petabytes of data, etc). They learn a very different craft than you do.
Your craft, the startup developer craft, is simple – “get things done”. The other parts of the craft, you have to earn.
(caveat – if you are building a startup focused on platform or tools being used by other developers, your craftsmanship should be excellent)
(disclaimer – I have nothing against facebook or google, they are full of friends of mine and other wonderful and smart ppl)
Who doesn’t love tech porn? And what’s better than an inside look at the architecture and tools that power a startup? That’s right, nothing. So we thought, why not put up our own little behind the scenes, and try and share a little bit about how we do what we do?
At Upverter, we’ve built the first ever web-based, the first ever collaborative, and the first ever community and reuse focused EDA tools. This meant re-thinking a lot of assumptions that went into building the existing tools. For example, clients and servers weren’t an afterthought, but instead a core part of our architecture. Collaboration was baked in from the start which also meant a whole new stack – borrowed heavily from guys like Google Wave, and Etherpad.
On the front-end, our pride and joy is what we call the sketch tool. Its more or less where we have spent the bulk of our development time over the last year – a large compiled javascript application that uses long polling to communicate with the API and Design Servers. When we started out to move these tools to the web, we knew that we would be building a big Javascript app. But we didn’t quite know what the app itself would look like and our choice of tech for the app itself has changed quite a bit over time… more on this later!
On the back-end, we run a slew of servers. When it comes to our servers, there was a bit of a grand plan when we started, but in reality they all came about very organically. As we needed to solve new problems and fill voids, we built new servers into the architecture. As it stands right now, we have the following:
Front-end web servers, which serve most of our pages and community content;
API & Design servers, which do most of the heavy lifting and allow for collaboration;
DB servers, which hold the datums; and
Background workers, which handle our background processing and batch jobs.
So let’s talk tech…
We use a lot of Linux (ub) (arch), both on our development workstations and all over our servers.
We use Python on the server side; but when we started out we did take a serious look at using Node.js () and Javascript. But at the time both Node and javascript just wern’t ready yet… But things have come a tremendously long way, and we might have made a different choice if we were beginning today.
We use nginx (http://nginx.org/) for our reverse proxy, load balancing and SSL termination.
We use Flask (http://flask.pocoo.org/) (which is a like Sinatra) for our Community and Front-end web servers. We started with Django, but it was just too full blown and we found ourselves rewriting it enough that it made sense to step a rung lower.
We use Tornado () for our API and design servers. We chose Tornado because it is amazingly good at serving these type of requests at break neck speed.
We built our background workers on Node.js so that we can run copies of the javascript client in the cloud saving us a ton of code duplication.
We do our internal communication through ZMQ (www.zeromq.org) on top of Google Protocol Buffers
We used MySQL () for both relational and KV data through a set of abstracted custom datastore procedures until very recently, when we switched our KV data over to Kyoto Tycoon ().
Our primary client the sketch tool is built in Javascript with the Google Closure Library () and Compiler ().
The client communicates with the servers via long polling through custom built RPC functions and server-side protocol buffers.
We draw the user interface with HTML5 and canvas (), through a custom drawing library which handles collisions and does damage based redrawing.
And we use soy templates for all of our DOM UI dialogs, prompts, pop-ups, etc.
We host on EC2 and handle our deployment through puppet master ().
Monitoring is done through a collection of OpsView/nagios, PingDom and Collectd.
Our development environment is very much a point of pride for us. We have a spent a lot of time making it possible for us to do some of the things we are trying to do from both the client and server sides and putting together a dev environment that allows our team to work efficiently within our architecture. We value testing, and we are fascists about clean and maintainable code.
We use git (obviously).
We have a headless Javascript unit test infrastructure built on top of QUnit () and Node.js
We have python unit tests built on top of nose ().
We run closure linting () and compiling set to the “CODE FACIEST” mode
We run a full suite of checks within buildbot () on every push to master
We also do code reviews on every push using Rietveld ().
We are 4-3-1 VIM vs. Text Edit vs. Text Mate.
We are 4-2-2 Linux vs. OSX vs. Windows 7.
We are 5-2-1 Android vs. iPhone vs. dumb phone.
If any of this sounds like we are on the right path, you should drop us a line. We are in Toronto, we’re solving very real-world, wicked problems, and we’re always hiring smart developers.
Back in May, Nat Friedman wrote about the tools used in setting up Xamarin. They include a great set of basic tools for getting a startup off the ground with very little investment. We have seen a lot of startups using a similar set of tools and I thought that we’d compile a list of the tools that we’re actively using (and some of the others we evaluated). There are the tools and blogs listed by Steve Blank that include many
Landing Pages
We’re big fans of WordPress at StartupNorth. We’ve powered StartupNorth on WP since the beginning. The combination of WordPress, Premise, and the WordPress MU Domain Mapping plugin is a pretty powerful combination for creating mutliple sites and landing pages to test your landing pages. But we’ve also developed a sweet spot for Vancouver’s Unbounce, it took us less than 5 minutes to have 2 landing pages and a domain set up. We’re big believers that you can use Adwords and Facebook Ads to quickly create a landing page to test ideas before writing a single line of code.
We primarily use Google Analytics and WordPress Stats for StartupNorth. We’ve been working with startups and using a KISSmetrics and Mixpanel to measure activity on their web properties and applications. Make sure you read Ash Maurya’s 3 Rules to Actionable Metrics to understand how the analytics can be used in combination with split testing and/or cohort analysis to better track your optimization before product/market fit (What do you measure before product/market fit? – check out Ash’s conversion funnel and metrics).
We haven’t been as proactive in building a mailing list for the StartupNorth community as we probably should have been. I’ve used have started using MailChimp because of the quick integration to GravityForms and WooFoo, but have had very positive experiences using both Campaign Monitor and Constant Contact.
What is amazing is that both of these companies are local to Toronto. We use WaveAccounting integrated with our bank account and PayPal for tracking expenses, billing, and financial operations. And we use Freshbooks to bill for sponsorships. They are a must have in our back office. What we’re missing is a really easy to use and integrated payroll system (I hear that it might be coming).
For full disclosure, I’m an advisor to TribeHR. It doesn’t change the fact that they rock. It is the easiest way to get an HR system in place. And there is no better way to get feedback and help employees improve than Rypple.
We are actively using Survey.IO to gather feedback from users about the state of StartupNorth. It helps us figure out the state of our product market-fit, if there is such a thing for a blog about Canadian startups, fill it out and help us be better.
We use Pivotal Tracker. We like them so much, we actively recruited them as a sponsor for StartupNorth. There are lots of other tools from project tools to issue tracking. Curious at what others are using.
We use Github Bronze for our project hosting. Most of the code we work on is PHP against MySQL (see WordPress), though we have additional apps in development like the StartupNorth Index (which will be moving to startupnorth.ca/index shortly) but all are LAMP.
Full disclosure: VMFarms is a sponsor of StartupNorth. However, their hosted VMs that are backed up and hot mirrored coupled with the outrageous “white glove” makes them a dead simple choice. We also use Rackspace Startups and EC2 for access to easy Linux and Windows VMs for development and testing environments.
I’ve been using Calliflower for conference calling. It’s $5/call for up-to 5 callers, or for $30/month unlimited minutes and >70 participants, it’s a great solution. It is not a replacement for a office phone system.
Google Voice and Skype have been the least expensive way as a Canadian startup to get a US phone number. This is great for me as an individual. However, this does not scale to an enterprise or an organization. I’ve been looking at Grasshopper, RingCentral and Toktumi, but I have yet to settle on a solution.
This part of the list is pretty much cribbed from Steve Blank’s list of tools for entrepreneurs. Go read it for a more comprehensive list of tools beyond the SEO/SEM listing included below.
I’m going to cover in the next post: discounted travel, conferences, business cards, design services, and other tricks for being relentless resourceful as a founder.
Along with the team at FreshBooks, StartupNorth is proud to support TechTalksTO. TechTalksTO previous put together a series of speakers at the Gladstone Hotel on West Queen West that feature some great talks about technology for Toronto developer community. We’re very happy to be a Sponsor and Media Sponsor of their upcoming conference event on August 13, 2011. It’s a great group of front-end, back-end and devops focused entrepreneurial technologists.
What a great way to spend a Saturday before flying out to GrowConf later in the week.
The Details
When: Saturday, August 13, 2011
Where: Toronto Underground Cinema, Spadina Avenue.
What: All-day conference + After-Party
Rick Olson (@technoweenie) – GitHub Alchemist and Rails core committer
Aaron Peterson (@metaxis) – Technical Evangelist at OpsCode, the creators of Chef
After Party
One of the best parts about our previous TechTalksTO events were the unofficial gatherings afterward. They were always a great opportunity to network and have some great conversations. We enjoy it so much, we figured for this event, we’d make it an official part of the day so your ticket will include admission, food, and drinks at the exclusive after-party Saturday evening to be held at a nearby establishment.
Tickets
Tickets are available NOW!. Alas, due to the size and scope of this undertaking, we have to charge actual money for tickets for this one but we think you’ll totally get your money’s worth. Tickets will be priced at $150, including admission to the after-party, which will also get you a couple of drinks and some food. We also plan to have some food and drinks available throughout the day at the conference venue.
I am always curious at what startups are using for the technology under the hood. It helps define the technologies that frontend and backend developers looking for jobs might start using. It helps understand what skills are being developed in the ecosystem and where competition or coopetition for talent may exist. And I am always curious at the emerging trends of technology adoption by local firms.
The online advertising landscape is changing rapidly, Chango is taking advantage of Real-Time Bidding to allow marketers to buy ads on the web in real-time. It means big data where time is critical because small fluctuations in price or users can have big impact on the overall effectiveness or cost. The best analogy is a stock market, where instead of stocks, marketers can buy billions of ads each day based on how they think that websites ad slots will perform for them. The cool part about RTB is that you can combine data to put the right ad in front of the right user. This requires huge data processing skills and Chango processes information on about 60,000 ads a second.
What does Chango do?
We are an ad-buying platform that is built for online marketers looking to find new customers using display (banner ads). Our main goal is to eliminate wasted marketing dollars through smarter targeting. Specifically we show ads to people based on what they’ve searched for in the past – a technique we are pioneering called “Search Retargeting“.
Unlike traditional ad networks we exclusively buy ads using real-time bidding, a new technology that is rapidly changing the Display advertising industry and allowing marketers to layer data on top of their media buys.
Chango is unique in that it has access to billions of search terms, billions of incoming ad impressions, and has devised machine learning techniques for combining data from different sources to deliver more efficient and better-targeted campaigns for our advertisers.
What does your product architecture look like?
Software development that deals with “Big Data” typically has two types of challenges: big volume, or low latency. Chango has to deal with both of these issues simultaneously.
We receive over 90,000 requests per second (and growing monthly) from our ad exchange and data partners at peak times of the day. To make matters more interesting, we have approximately 80ms to respond to any incoming bid requests. Unfortunately 30-50ms of this limit is used up because of unavoidable network latency, this leaves us only 30ms to process each bid request. As a result, we have to constantly optimize our bidder subsystem around this challenge.
What are some of the tools and technologies you use?
Open Source is at the heart of our technology stack. Python is the common thread that binds all of our subsystems. Our entire infrastructure is written in Python. We use a (modified version of) the Tornado Web Server for our realtime servers, and Django for the front-end.
When dealing with super fast response times it’s critical to have a super-fast datastore at your disposal. That’s why we use a NoSQL database technology based on the proven memcached server.
The unsung open source hero of our infrastructure is HAProxy. HAProxy handles our load-balancing across the board. We use the “keepalived” feature of Linux to keep these servers in high-availability mode.
As far as our architecture is concerned we try to not have any major dependencies on any specific features of the third-party systems we choose. They just happen to work well for our current environment.
How did you get here?
Scalability is as much an art as it’s a science, but most importantly, it’s about keeping things simple. It took years of hard work, and careful tuning and measurement to arrive at where we are.
Originally we used Java for all server-side development and Ruby on Rails for front end development. The thinking was that we needed a rock-solid language for server architecture and a rapid development environment for front-end work. This concept served us well for a little while; however, in early 2010 we realized that Java was drastically slowing down our ability to iterate quickly and effectively. A single feature was taking days to build, test and deploy.
We bit the bullet and rebuilt the platform entirely in Python and it was probably the best decision we ever made. Not only do we have a consistent language across front end and server side development but it has enabled us to rapidly add features or test new ideas. We are fortunate to have access to the fantastic ecosystem created by the Python community.
Where do you host?
Our first approach back in 2009 was to leverage Amazon Web Services EC2 as a scaleable and cheap way to prototype the platform. That served us well for a while; however, the shared virtual environment meant that we had wildly variable server resources.
We shifted to Hosting.com knowing that we ultimately needed our own equipment and if we wanted a VM environment we would need to set one up ourselves. While Hosting.com provided good support at the time we wanted the rapid provisioning we were used to at EC2 with the power of dedicated hosting.
Ultimately we chose SoftLayer as our hosting provider of choice. SoftLayer offers a VM environment in addition with their “express” service that allows us to get 10s of new servers provisioned in about 3-4 hours! They have been extremely good about allowing us to occasionally provision a whole new parallel cluster as we do capacity planning.
How do you monitor your systems?
Monitoring is done through a combination of Nagios, PagerDuty, and Circonus among others. We have also built a real-time data visualization system that let’s us monitor both infrastructure, and campaign performance. We use this dashboard as our own NOC system hooked up to a TV that is mounted right in the engineering area!
What are your biggest development challenges?
We’ve got two distinct challenges. Our real-time, data processing, and systems engineering teams deal with problems of scalability and big data. As our business continues to grow, we need to re-examine our infrastructure and design choices. We have a very healthy culture of team collaboration, code-review, and refactoring.
Our Dashboard team has a different set of challenges. Our self-serve ad platform is very much like Adwords in that marketers can put down a credit card and launch a campaign themselves. We need to make this an extremely user-friendly system, while keeping it powerful enough to enable people to perform sophisticated reporting and campaign optimization.
How do you win?
The Chango business is all about putting the right ad in front of the right user at the right time. We made an early decision that data you know about a user (ie. search data) is only effective if it is combined with a proprietary bidding engine that can make decisions in real time.
Almost every DSP, data provider or ad network out there today does this by storing information about users in a client-browser “cookie”. They call this a user segment. The problem with user segments is that they are pre-computed and stored within the users browser. If you decide half way through your campaign that you need to adjust your audience (due to lack of performance) there is no way to do so since the information is hard coded in the users browser. The only option is to continue serving ads to this under-performing group of users and wait for that cookie to expire (typically 30 days).
At Chango we’ve decided to make everything real-time, including our decision about who to bid on, and how much to bid. Nothing about the user is pre-computed. So the Chango ‘cookie’ contains nothing more than a unique identifier that anonymously points to all the raw data we know about that person in our database.
Chango is hiring!
Python Developers! There are multiple open positions in Toronto, New York & San Francisco. But if I could have anything it would be talented developers that either know python or want to learn Python.
Interested in being profiled in our Under the Hood series, we are actively looking for Canadian startups building “interesting” technologies and solving “interesting” problems. Contact me by completing your initial Under the Hood submission.
Since I wrote the interview with Farhan of Xtreme Labs on Pair Programming, I have exchanged a number of emails on pair programming & developer productivity with some of Canada’s best developers. Most of them are pure gold that I am posting raw and unedited.
———————————————————————————————
“The main thing we like to stress on development is Test Driven development. We just find that it both ensures all the code is well tested, but also guides the architecture in a way that really improves maintainability. We use pairing in our interview process and also like to pair for the first couple of weeks to get developers up to speed, but after that we tend to split up work for the day and then attack separately.” – Jesse Miller, CEO of attachments.me
———————————————————————————————
“Whether to use pair programming or alone model really depends on the team personality and the kind of culture the leaders want to shape up. Some people do work better alone, and some prefer more teamwork.
Personally, I usually value teamwork more than individual results. At the end of the day, individual work will have to be integrated to form the end product, which is what really matters. There are too many examples where people think they got great ideas, but end up in epic failure during implementation. It is a result of not doing deep enough design validation. Coding alone is definitely more effective when you got the design details flushed out. Personally, when I am coding alone, I still ask for 2nd or 3rd opinions on a day to day basis. It forces me to think through the design again and again, which helps to refine my idea further. It is all about attitude. Never believe that you got it right until see the end product.
Another interesting thing is “pair troubleshooting” is more effective. When you’re trying to debug in an extremely stressed environment, it is always good to have some mental support and another pair of eyes.” – John Yuen, heads up dev for Fixmo
———————————————————————————————
“I think that pair programming does actually increase productivity, in most cases, and there are two reasons for this:
1. It keeps programmers from doing other things:private email, browsing, day dreaming, etc. I doubt that the average programmer actually spends even 50% of their time doing their job. If you can hire two developers, and get the work of one full programmer out of them, then I think, in a lot of cases, that would be an improvement. However, you could get almost the same result by pairing each
programmer with their own full-time low-cost supervisor who sits beside them to make sure that they’re actually working. This would be a lot cheaper and almost as productive, but the developers would find it insulting. This isn’t always the case of course; there are certainly many self-motivated, disciplined, highly-focused and productive developers who don’t need supervision.
2. Studies have shown that programmer productivity is limited, not by the time it takes developers to write code, but by the time it takes them to detect and correct errors. The limiting metric isn’t lines-of-code-per-hour, but rather, errors-debugged-per-hour. This is why Java is more productive than C++, despite the two languages being of similar expressive levels, Java’s runtime exceptions make it quicker to detect and recover from many times of errors, and garbage collection completely eliminates many other types. This is where pair-programming comes into play, if someone looking over your shoulder can spot an error as soon as it’s created, then that can lead to a lot of time savings. Maybe it would have only taken you two minutes longer to detect and fix a particular bug, but then gain, maybe it would have taken a week. Your partner wouldn’t need to early-detect very many problems in order to justify the overhead.
Maybe one developer sitting in a control booth, like a security guard, monitoring the work of half-a-dozen developers all at once, would be a good compromise between pair and pair-less programming.
Yes, it’s true that ACM programming contests are performed on only one computer, but you rarely have more than one person working on the same problem. While one is typing in and debugging their solution, the other programmers are working with pencil and paper solving their own problems (with occasional help from their team mates as required).
People are more important than any process or any tool. The best process is to hire the best people and a better developer is the best tool.
I half agree with what the original article says about hero developers. They say that you don’t need them, and that you’re better off with the a consistently productive 9-to-5’er. I think they’re
confusing heroes with heroics. I think that you do need heroes, but you don’t need the heroics of pulling all-nighters. Instead, you need the heroics of consistently creative and proactive problem solving which keeps you efficient, effective, and ahead of the game, day in, and day out. Effort should not be confused with results. I think they have the wrong definition of heroics, which leads to the wrong definition for heroes.” – Kevin Greer, Framework Lead at Redknee.
———————————————————————————————
Point #2 from Kevin Greer’s comments is widely unappreciated. Most people judge productivity by features and the “to do” list – when in reality it is not the to do list of features that usually makes projects late, it is the killer game of bug squashing, especially as you get to the stressful end of project delivery. Processes that reduce the upfront bug rate or increase the bug solving rate, even at the expense of adding features pay off quickly. This is were pair programming, test driven development, code reviews, architecture chats, etc all come into play. And why “pair troubleshooting” or other tools to debug problems fast are important. And also, use the damn debugger, it is there to make solving bugs faster.
I’d love to hear from more of the programming community in the comments. Tell us how you supersize your development productivity.
Last week Tim Ferris, of 4 Hour Work Week fame interviewed the guys from Pivotal Labs and learnt the secret of their success – pair programming! It was a very thought provoking piece on how to get stuff done.
Well here in Canada we have our own super hot mobile dev shop, Xtreme Labs, @xtremelabs, who are absolutely killing it, picking up huge customers like GroupOn. The guys at Xtreme Labs are also infamous amongst the tech scene for being pushers of pair programming.
Now at Peek, where I’m CTO, we are all “Lonely Hero Programming” all the time. Our “bus count” is precisely 1. If somebody died tomorrow you know what I’d do? I’d hire somebody new!
The Best Way To Program – By Vinoth Chandar, Some Rights Reserved
So, being from this totally opposite world to pair programming, I wanted to learn a bit more about why a company would choose pair programming, and I thought I’d start up a friendly dialogue with Farhan Thawar of Xtreme Labs to gather this thoughts.
Dan: Hey Farhan, how are things? I’d love to hear more about how you guys program at Xtreme. How are you setup?
Farhan: Our work environment is what I call an Agile Team room. Super low cubes, everyone in one area, with programmers focussed in a particular technology seated close to each other (so iOS devs sit together, Android devs sit together, etc.).
Dan: I’m going to start right off the bat with some basic math. If I have two people solving one problem versus two people solving two problems, doesn’t that automatically make me less productive? You know 1+1=2? How can I as a lean running startup afford to be half productive???? You guys have to pay your 2x workforce in real cash, right?
Farhan: So the simple math is misleading, i.e. that 2 programmers working on the same task is less effective. For example, most agile shops typically get 4-5 hours of coding done per day. The rest is spent in email, meetings, Facebook, Twitter, Youtube, reading blogs, etc. With pairing, we can focus our folks for 8-9 hours together to get a full days work done. Yes, it’s 8-9 hours / 2 = 4-4.5 hours of coding each, so at a minimum it’s the same hours. However, we have instant robustness (bus count), higher quality (i.e. less hacking), instant knowledge transfer (two people touch every line of code).
Dan: I am a programming god. I go on tears for 2-4 weeks where I am an absolute machine. Knuth himself would have a hard time keeping up with me as I weave around in my language du jour (i.e. Haskell, gods only program in Haskell.. the language of the heavens). The last thing I need is some assclown developer tethered to my leg slowing me down. How do you handle god coders like me in pair programming?
Farhan: Lol, badass programmers get better by pairing, just one example: http://cycle-gap.blogspot.com/2007/09/extreme-pair-programming-guy-steele-and.html. The reason is that it’s an intense learning environment, much like apprenticeship in the old days (you apprenticed to learn your trade). There’s no better way to get better and faster at your trade than to work with someone else who is as smart as you are.
Dan: How does this work if you work on a lot of projects? What if you are doing maintenance on a few projects as well as new dev’t on another project? We have lots of guys who probably touch 2-3 projects on any given day? Do they have to find their pairing buddy for each project every time?
Farhan: So the key is to not swap projects often. Almost 100% of our pairs are on a single project for the entire week. So week to week things my change (most time they don’t), but our devs are focused just on one project, on one platform. We do have R&D pairs and a Sustained Engineering pair who focus on multiple updates and fixes, but they aren’t dedicated full-time to any project. Folks also don’t find a pair, they are allocated before the week starts. We take tons of feedback on pairings and it’s very easy to see pairs that aren’t working well (they don’t talk much, or one person is driving the whole time)
Dan: In a debate in my company a great coder I know basically said something like this “don’t most coders do pair whiteboarding/pair designing anyways? By the time it comes to actually coding, isn’t that the trivial part?” Lets pretend this is a law office or consultancy shop, would you pair (i.e. do teamwork) at the strategy phase/problem solving phase or would you pair during the document creation phase?
Farhan: So the coding isn’t the trivial part, as the end product is based on the code. It’s almost like saying the ingredients and the recipe are the hard part but the cooking is trivial. It’s not (and I know, cause I can’t cook for shit). You want to write code that is elegant, understandable, maintainable, etc. and pairing forces that as two people have to understand what is going on at all times.
Dan: Have you guys ever tried doing like a race? Put two coders doing 1+1 vs two coders pairing on the same project? Should we try to sponsor this type of event?
Farhan: Bring it. We’d love this. Don’t forget, ACM programming competitions only use one computer 🙂
Process Matters
So here’s my take overall. Process matters. Even for dev teams of 1-3 people selecting the right language, technology, tool chain, and software process make a big difference in productivity and quality. Software engineering has improved by leaps and bounds over the past 5-10 years. Here are some of the best changes:
Opensource software and the explosion of re-usable software components
Iterations and demos
Continuous Integration, build servers and automated test suites
Continuous Deployments, i.e. the newer devops movement
Explosion of tools & infrastructure in the build, test, deploy area (think of tools like Heroku, Chef, Capistrano, build tools, etc)
Test driven development, test automation & significantly improved testing frameworks
(Sometimes I write in ANSI C on proprietary embedded plaforms, where the tool chain and quantity of re-usable code is a fraction of what I want. You take for granted how much Java, Ruby on Rails, Python, PHP, etc have been built up.)
These days, I rarely come across startups that haven’t adopted most of the above to a certain degree. And pair programming, truthfully its just not widely adopted compared to the list above. To me pair programming seems to be an overkill solution to solve the real programming productivity problem – communication. Engineers are classically trained to solve problems independently (though this is changing at the university level finally). Dr Amol Sarva, CEO at Peek, always mentored me with the following advice on communication, picked up during formal structured problem solving training at McKinsey.
“Basically there are three stages of any problem where one can communicate and practice team work:
1. Structuring – during framing of the problem. What are the steps I am going to take to solve this problem? Read the internet, talk to an expert, write quick hacks to test it, etc.
2. Solving – during solving the problem. “Hey, I did this and this and here are the raw results, interesting, right?”
3. Synthesis – communication while synthesizing the results of problem solving. “Hey guys, I solved this, here’s how. Code is checked in.”
Engineers classically wait until synthesis to solve the problem. Which is too late. Others, (e.g. your classic ice-breaking consultant) do too much communication during structuring.”
Pair programming forces engineers to communicate much earlier on in the problem solving process. Which is good! It also forces them to communicate across the whole problem solving spectrum from structuring to synthesis. Also good! But god it sure feels like an inefficient oversolve to the problem. Managers can bake in communication to their dev process without forcing pair programming in my opinion.
Having said all that, I am always willing to try something new (good engineering demands experimentation and learning), so we are going to try pairing it up on a project at Peek and see if it works well or not. I’ll report back the results.
I’d love to hear what others think. Is pair programming the real deal, have you found it to be more efficient in getting your code on? Or do you like to pound it out solo with headphones and your favourite hoodie?
We are pleased to be supporting Hack/Reduce Toronto. The rise of real-time computing, distributed sensors and big data have provided the ground work for development of a way to distributed the processing of these emerging large data sets across a cluster of computers. There are lots of Toronto and global companies leveraging the processing and analysis of large data sets to discover unique relationships in their data (Backtype, Postrank, BuzzData, Attachments.me, Google, and others. This is a wonderful opportunity for technical cofounders to get experience leveraging Map/Reduce, Hadoop and the shared expertise of local experts with some hands-on learning about big data.
Hack/Reduce is a free one-day big data hackathon. The goal is to extract valuable information from large datasets and learn how to work with big data. The event brings together Developers, Companies, Entrepreneurs and Students interested in Big Data.
Provided:
Free access to Amazon EC2 clusters that can be scaled up according to your needs.
Pre-loaded datasets (participants are encouraged to suggest datasets)
Introduction to Hadoop and Map/Reduce and the infrastrucure
Support from Hadoop and Map/Reduce experts
Food and drinks
At the end of the event, participating teams and developers get to present what they have done, what they learned and what problems they faced. It’s an opportunity to develop something great, learn Hadoop MapReduce and meet people interested in big data.
Who is it for?
Developers, researchers and students in big data or interested in working with big data. The best thing is if you have something you want to get done that requires a lot of computing power. Alternatively, you can come to learn to use Hadoop. Basically Hack/Reduce is about developers, working with new people, pizza, unlimited computing power and large data sets.
Some rights reserved by TonyFelgueiras
