The Evolution of the StockTwits Technical Architecture
- October 25th, 2011
Let me start by saying that StockTwits is not Twitter. Today, StockTwits registered users/member share information to Twitter, and from Twitter to StockTwits. Several years ago, we created StockTwits to curate real-time financial information coming from Twitter. This technical and product direction lasted only a few months, and then quickly changed. This is the story of how we moved from simply a site displaying Twitter “Tweets”, to an independent idea network built on our scalable platform.
Twitter was a two year old company at the time, and while they were focused on scale, other companies like ourselves were dreaming up ways to use the service and data to jump start our ideas. Soren Macbeth and Howard Lindzon had created a language to talk about public companies, using what I like to call a cashtag ($DELL, $HPQ, $GRPN). It’s similar to a reply or hashtag in that it was created by us to talk about public companies on Twitter. This meant that when a user of Twitter talked about a public company using the cashtag we would grab the tweet from the Twitter API and aggregate the conversation per ticker. It was a simple concept and we built the application using the Twitter API and data.
Did I mention in 2008 Twitter was growing at a rapid pace, and had some major scaling issues? Well we started to notice the issues daily when dealing with the Twitter API. Our application was 100% built using all aspects of the Twitter API. Our user system was the Twitter authentication. The messaging was 100% Twitter Tweets. The aggregation would happen pulling from the Twitter API. The only thing that wasn’t Twitter was the message processing and aggregation that would happen in the backend! It was amazing what we had built in less than a month all off the Twitter plumbing and architecture. They allowed us to try something fast using a great data set and functionality without having to worry about a lot of the plumbing.
While it was amazing being a Twitter application at that time, it didn’t come without it’s troubles. Twitter had been experiencing a lot of outages and “fail whales”. Our developer, Dustin Schneider, deserves a lot of credit for becoming fluent in the Twitter API, and all the error codes. On some days our site would look like it was down or non functioning, when in reality the servers and application were up and running — the problem tended to be Twitter’s API. Of course when this would happen, we would get calls from our growing community of StockTwits faithful. They would be frustrated and disgruntled as to why our site wasn’t working.
People had grown very accustomed to easily sharing their ideas in real-time with the new people they had met on StockTwits and reading through all the messages on the hot stocks that day, so you can see why some might get upset when things were not available.
As issues progressed, we made the decision to build our own platform similar to Twitter, but completely independent of them. We had to do this to best serve our growing finance community. The community wanted the reliability of contributing ideas and reading them that only owning our own platform could provide.
We set out on the hard task of creating our own messaging platform but, unlike Twitter, with only the real-time investor and trader in mind. We added additional meta data into the messages like what stock members were talking about, the price of the stock at that time, who members were talking to and much more trader/investor specific data. Charts and stock pricing were added on the front-end for users to reference.
We created our own authentication and profiles with trader-specific questions and data, and added integrations with other social networks like Facebook and LinkedIn. StockTwits now has its own financial social graph, interest graph, and influencer scores. We created, and are still creating, many discovery tools to help our users find the right stocks, as well as like-minded people.
In addition to our own on-network experiences, we created our own API that we use to power our own apps like iPhone and Android, as well as select distribution partners like Yahoo! Finance, Bing, Reuters, CNN Money and others. We have Fortune 100 companies paying for our Investor Relations product as well that runs off the same platform.
We are now our own real-time social network for the financial community. The place where investors, traders, and companies come to share real time ideas. Today I am proud to say we are experiencing double digit growth on a continuing basis, because of our platform and financial focus. I am also proud to say that my team has built an independent platform, that is not reliant on Twitter. It is our platform, it is stable and scalable enough to handle the heavy requirements of the financial markets.
The information in this blog post represents my own opinions and does not contain a recommendation for any particular security or investment. I or my affiliates may hold positions or other interests in securities mentioned in the Blog, please see my Disclaimer page for my full disclaimer.blog comments powered by Disqus
StockTwits® is an open, community-powered idea and information service for investments. Users can eavesdrop on traders and investors, or contribute to the conversation and build their reputation as savvy market wizards. The service takes financial related data and structures it by stock, user, reputation, etc. More »