Once again I took advantage of the amazing meetups in Austin to listen to Wesley Chun from Google discuss their App Engine PaaS as well as details on other cloud products. The session did get very technical as the meetup was for the Google Developers Group however I will present the information at a bit higher level since I am not a Django programmer and I “tuned” out that part. Also, I want to thank Spanning for hosting the event in their downtown offices which are impressive.
The event focused on the Google Cloud solutions with an emphasis on their App Engine but the overall theme of the night was the constant reminder that Google does thing at a massive scale. Here are some data points from late 2011 they presented to give you an idea of what scale means to Google:
- Size of Google search index – 100 Million Gigabytes
- Number of YouTube video hours uploaded per minute – 72 hours
- Google App Engine # of active developers per month – 150,000+
- Google App Engine # of active applications per month – 200,000+
- Google App Engine applications generate 2 billion page views per day
Wesley also presented the basic philosophy of Google around their cloud solutions – take their internal solutions and open them up to the world. In the SaaS space they offer Google Apps and in the PaaS space they offer Google App Engine with a recent announcement a few months back of Google Compute Engine for IaaS. More on Google Compute Engine later in this post.
As for App Engine, the focus is on three criteria delivering “true elasticity”:
- Easy Build
- Easy Manage
- Easy Scale
I am skipping the PaaS overview portion of the discussion and send you off to Wesley’s post on the cloud segment. App Engine places your application in its own Sandbox where it cannot interfere or be interfered with from other applications on the platform. A set of APIs/Services are available for developers to leverage when writing applications for App Engine to ensure your application operates properly within the infrastructure. Some of the API/Services presented were Memcache, Mail, XMPP, Datastore, Images, User Services,…
An interesting comment from Wesley on performance of applications in Google App Engine is worth repeating, applications must respond to a customer in less than 60 seconds or that instance will be immediately shut down. In fact, the faster your application responds to the users, the better your application will perform in terms of getting faster scale-up, etc. The longer your application takes, the less likely your application will receive additional infrastructure to support it. The message is pretty clear that your application needs to be incredibly responsive and run efficiently to take advantage of the scaling capabilities of App Engine.
Some customers using App Engine included Best Buy, Evite, Direct TV, Forbes, MTV, Buddy Poke, Pulse mobile app, and Gigya. Here are some amazing stats from these customers to give you perspective on what scale means to Google.
- Buddy Poke – More than 62 million users; ~ 10% of user base runs the application daily; App Engine Datastore holds at least 62 million data objects
- Gigya – Create applications for one-off events that have huge instant scale potential; royal wedding in UK saw spike from 0 page requests per second to 1,600 page requests per second in a 3 hour period
- Pulse – serves 100 million requests per day
Finally, I wanted to comment on Google’s statements about their Google Compute Engine which was launched to a private beta in May 2012. The target for this IaaS is not simple web applications or random VMs but rather significantly large (hard-core) compute intensive applications requiring thousands of cores and super-fast inter-VM communication. Google has created a private network for the VMs that are separated from other users on the system to give high performance VM communication a priority. This focus on a public IaaS cloud is different than what I have seen from vendors such as Amazon, VMware, etc. My take is that they are building a high performance computing (HPC) public cloud to handle the truly big, massive computationally intensive processing.
Google also talked about other cloud solutions they offer including Cloud Store, Cloud SQL, Big Query, Google Prediction, Fusion Tables, and Course Builder MOOC. I was not aware of some of these products and suggest you spend some time on Google’s cloud sites to learn more about some interesting products.