Under the Hood: The Technical Setup of Upverter

Editor’s note: This is a cross post from the Upverter blog written by Zak Homuth (LinkedIn, @zakhomuth, Github). Follow him on Twitter @zakhomuth. This post was originally published on August 1, 2011, I was just negligent in posting it.

Who doesn’t love tech porn? And what’s better than an inside look at the architecture and tools that power a startup? That’s right, nothing. So we thought, why not put up our own little behind the scenes, and try and share a little bit about how we do what we do?

At Upverter, we’ve built the first ever web-based, the first ever collaborative, and the first ever community and reuse focused EDA tools. This meant re-thinking a lot of assumptions that went into building the existing tools. For example, clients and servers weren’t an afterthought, but instead a core part of our architecture. Collaboration was baked in from the start which also meant a whole new stack – borrowed heavily from guys like Google Wave, and Etherpad.

http://en.wikipedia.org/wiki/Apache_Wave
http://code.google.com/p/etherpad/
http://techblog.gomockingbird.com/archive/5/2010

 

Apache-wave

On the front-end, our pride and joy is what we call the sketch tool. Its more or less where we have spent the bulk of our development time over the last year – a large compiled javascript application that uses long polling to communicate with the API and Design Servers. When we started out to move these tools to the web, we knew that we would be building a big Javascript app. But we didn’t quite know what the app itself would look like and our choice of tech for the app itself has changed quite a bit over time… more on this later!

On the back-end, we run a slew of servers. When it comes to our servers, there was a bit of a grand plan when we started, but in reality they all came about very organically. As we needed to solve new problems and fill voids, we built new servers into the architecture. As it stands right now, we have the following:

  • Front-end web servers, which serve most of our pages and community content;
  • API & Design servers, which do most of the heavy lifting and allow for collaboration;
  • DB servers, which hold the datums; and
  • Background workers, which handle our background processing and batch jobs.

 

 

So let’s talk tech…

  • We use a lot of Linux (ub) (arch), both on our development workstations and all over our servers.
  • We use Python on the server side; but when we started out we did take a serious look at using Node.js () and Javascript. But at the time both Node and javascript just wern’t ready yet… But things have come a tremendously long way, and we might have made a different choice if we were beginning today.
  • We use nginx (http://nginx.org/) for our reverse proxy, load balancing and SSL termination.
  • We use Flask (http://flask.pocoo.org/) (which is a like Sinatra) for our Community and Front-end web servers. We started with Django, but it was just too full blown and we found ourselves rewriting it enough that it made sense to step a rung lower.
  • We use Tornado () for our API and design servers. We chose Tornado because it is amazingly good at serving these type of requests at break neck speed.
  • We built our background workers on Node.js so that we can run copies of the javascript client in the cloud saving us a ton of code duplication.
  • We do our internal communication through ZMQ (www.zeromq.org) on top of Google Protocol Buffers
  • Our external communication is also done through our custom RPC javascript again mapped onto Protocol Buffers. http://code.google.com/apis/protocolbuffers/docs/overview.html/
  • We used MySQL () for both relational and KV data through a set of abstracted custom datastore procedures until very recently, when we switched our KV data over to Kyoto Tycoon ().
  • Our primary client the sketch tool is built in Javascript with the Google Closure Library () and Compiler ().
  • The client communicates with the servers via long polling through custom built RPC functions and server-side protocol buffers.
  • We draw the user interface with HTML5 and canvas (), through a custom drawing library which handles collisions and does damage based redrawing.
  • And we use soy templates for all of our DOM UI dialogs, prompts, pop-ups, etc.
  • We host on EC2 and handle our deployment through puppet master ().
  • Monitoring is done through a collection of OpsView/nagios, PingDom and Collectd.

Our development environment is very much a point of pride for us. We have a spent a lot of time making it possible for us to do some of the things we are trying to do from both the client and server sides and putting together a dev environment that allows our team to work efficiently within our architecture. We value testing, and we are fascists about clean and maintainable code.

  • We use git (obviously).
  • We have a headless Javascript unit test infrastructure built on top of QUnit () and Node.js
  • We have python unit tests built on top of nose ().
  • We run closure linting () and compiling set to the “CODE FACIEST” mode
  • We run a full suite of checks within buildbot () on every push to master
  • We also do code reviews on every push using Rietveld ().
  • We are 4-3-1 VIM vs. Text Edit vs. Text Mate.
  • We are 4-2-2 Linux vs. OSX vs. Windows 7.
  • We are 5-2-1 Android vs. iPhone vs. dumb phone.

If any of this sounds like we are on the right path, you should drop us a line. We are in Toronto, we’re solving very real-world, wicked problems, and we’re always hiring smart developers.

Reference

Editor’s note: This is a cross post from the Upverter blog written by Zak Homuth (LinkedIn, @zakhomuthGithub). Follow him on Twitter @zakhomuth. This post was originally published on August 1, 2011, I was just negligent in posting it.

Lean Startup Tools

Some rights reserved by splorp - CC-BY-NC-ND
AttributionNoncommercialNo Derivative Works
 Some rights reserved by splorp

Back in May, Nat Friedman wrote about the tools used in setting up Xamarin. They include a great set of basic tools for getting a startup off the ground with very little investment. We have seen a lot of startups using a similar set of tools and I thought that we’d compile a list of the tools that we’re actively using (and some of the others we evaluated). There are the tools and blogs listed by Steve Blank that include many

Landing Pages

We’re big fans of WordPress at StartupNorth. We’ve powered StartupNorth on WP since the beginning. The combination of WordPress, Premise, and the WordPress MU Domain Mapping plugin is a pretty powerful combination for creating mutliple sites and landing pages to test your landing pages. But we’ve also developed a sweet spot for Vancouver’s Unbounce, it took us less than 5 minutes to have 2 landing pages and a domain set up. We’re big believers that you can use Adwords and Facebook Ads to quickly create a landing page to test ideas before writing a single line of code.

Analytics

We primarily use Google Analytics and WordPress Stats for StartupNorth. We’ve been working with startups and using a KISSmetrics and Mixpanel to measure activity on their web properties and applications. Make sure you read Ash Maurya’s 3 Rules to Actionable Metrics to understand how the analytics can be used in combination with split testing and/or cohort analysis to better track your optimization before product/market fit (What do you measure before product/market fit? – check out Ash’s conversion funnel and metrics).

Mailing Lists

We haven’t been as proactive in building a mailing list for the StartupNorth community as we probably should have been. I’ve used have started using MailChimp because of the quick integration to GravityForms and WooFoo, but have had very positive experiences using both Campaign Monitor and Constant Contact.

Billing and Accounting

What is amazing is that both of these companies are local to Toronto. We use WaveAccounting integrated with our bank account and PayPal for tracking expenses, billing, and financial operations. And we use Freshbooks to bill for sponsorships. They are a must have in our back office. What we’re missing is a really easy to use and integrated payroll system (I hear that it might be coming).

Human Resources

For full disclosure, I’m an advisor to TribeHR. It doesn’t change the fact that they rock. It is the easiest way to get an HR system in place. And there is no better way to get feedback and help employees improve than Rypple.

Surveys and Feedback

We are actively using Survey.IO to gather feedback from users about the state of StartupNorth. It helps us figure out the state of our product market-fit, if there is such a thing for a blog about Canadian startups, fill it out and help us be better.

Project Tracking

We use Pivotal Tracker. We like them so much, we actively recruited them as a sponsor for StartupNorth. There are lots of other tools from project tools to issue tracking. Curious at what others are using.

Source Control

We use Github Bronze for our project hosting. Most of the code we work on is PHP against MySQL (see WordPress), though we have additional apps in development like the StartupNorth Index (which will be moving to startupnorth.ca/index shortly) but all are LAMP.

Hosting

Full disclosure: VMFarms is a sponsor of StartupNorth. However, their hosted VMs that are backed up and hot mirrored coupled with the outrageous “white glove” makes them a dead simple choice. We also use Rackspace Startups and EC2 for access to easy Linux and Windows VMs for development and testing environments.

Customer Relationship Management

We don’t have any strong recommendations. There are platforms like Salesforce that are fantastic and sales teams are used to. There is Highrise which is broadly supported with a lot of 3rd party tools. But so far, neither of these has been the clear winner for us. There is a great Quora question about “What is the best CRM for startups” that lists SFDC, SugarCRM and Highrise. There are a lot of choices for CRM including NimbleInsightlyWoosabiCapsuleSolve360,AppPlaneBatchbookPipelineDealsTactileCRMZohoCRM and many others.

Conferencing, Screen Sharing & Telecommunications

I’ve been using Calliflower for conference calling. It’s $5/call for up-to 5 callers, or for $30/month unlimited minutes and >70 participants, it’s a great solution. It is not a replacement for a office phone system.

Google Voice and Skype have been the least expensive way as a Canadian startup to get a US phone number. This is great for me as an individual. However, this does not scale to an enterprise or an organization. I’ve been looking at Grasshopper, RingCentral and Toktumi, but I have yet to settle on a solution.

SEO & SEM Tools

This part of the list is pretty much cribbed from Steve Blank’s list of tools for entrepreneurs. Go read it for a more comprehensive list of tools beyond the SEO/SEM listing included below.

What are we missing?

I’m going to cover in the next post: discounted travel, conferences, business cards, design services, and other tricks for being relentless resourceful as a founder.

There are a lot of online tools that startups are using to make or break their business. And there is a lot missing, monitoring like NewRelic, PagerDuty, Pingdom and Blame Stella for example. But I’m curious what are the indispensable tools being used at iStopOver.com, HighScoreHouse, CommunityLend, Idee/Tineye, Massive Damage, Empire Avenue, Indochino, Lymbix, Hootsuite, AdParlor, Locationary, Chango and others. What are you using? What gives you the edge in quickly and effectively gathering feedback to test your hypotheses?

TechTalksTO Underground

Tech Talks TOAlong with the team at FreshBooks, StartupNorth is proud to support TechTalksTO. TechTalksTO previous put together a series of speakers at the Gladstone Hotel on West Queen West that feature some great talks about technology for Toronto developer community. We’re very happy to be a Sponsor and Media Sponsor of their upcoming conference event on August 13, 2011. It’s a great group of front-end, back-end and devops focused entrepreneurial technologists.

What a great way to spend a Saturday before flying out to GrowConf later in the week.

The Details

When: Saturday, August 13, 2011
Where:  Toronto Underground Cinema, Spadina Avenue.
What:  All-day conference + After-Party


Speaker Lineup

After Party

One of the best parts about our previous TechTalksTO events were the unofficial gatherings afterward. They were always a great opportunity to network and have some great conversations. We enjoy it so much, we figured for this event, we’d make it an official part of the day so your ticket will include admission, food, and drinks at the exclusive after-party Saturday evening to be held at a nearby establishment.

Tickets

Tickets are available NOW! Alas, due to the size and scope of this undertaking, we have to charge actual money for tickets for this one but we think you’ll totally get your money’s worth. Tickets will be priced at $150, including admission to the after-party, which will also get you a couple of drinks and some food. We also plan to have some food and drinks available throughout the day at the conference venue.