OSCON Day3 – Google App Engine

I have seen several talks on Google App Engine before, but have still not used it in anger, so this talk, Introduction to Google App Engine, acted as a refresher. It was given by Ikai Lan, a software engineer working for the Developer Programs groups at Google.

Google App Engine is a way to run your applications on Google infrastructure. You push your code to App Engine and it gets scaled out depending on how many instances you need.

My notes from the talk are below, but you can find the full slides here or here.

LAMP Setups

The motivation for using Google App Engine is due to the limitations or constraints of a typical LAMP (Linux, Apache, MySQL and Perl/PHP/Python) setup.

1) Single Server

A single server running the LAMP stack is not scalable and has a single point of failure.

2) Separate web & database server

The next step would be to move the Apache server and the database to separate servers. You may get better performance but this is still not a highly scalable solution and you now have two single points of failure.

3) Multiple web servers

You can provide a more scalable solution by adding multiple Apache web servers, but now you need load balancing and the database remains as a single point of failure.
In order to utilize the multiple web servers, you can use round robin load balancing. Register the web server IPs with DNS; DNS record is cached with a TTL (Time to Live). But the TTL takes time to propagate down and may not be respected, so if a web server goes does, you can still have problems. Additionally, the database remains a single point of failure.

4) Master Slave Database

The next approach is to use a Master Slave database. Note that this approach was also discussed at the Database Scalability Patterns talk, and the Database Sharding talk. This Master Slave database approach improves read throughput, but the Master is now a single point of failure for writes and may die before replication takes place.

5) Partitioned Database

So, you can migrate to a partitioned database where you will (hopefully) have better read and write throughput, but where you now have more machines and more management and you may need to update your application logic, queries and schema.

Google App Engine to the rescue

So, in order to avoid all the hassle of above, Google propose using the Google App Engine.

Cloud Computing landscape

App Engine fits into the Cloud Computing landscape as follows:
Software as a Service: Google Docs, saleforce.com
Platform as a Service: Google App Engine, force.com
Infrastructure as a Service: Amazon Web Services, Rackspace, VMWare

App Server

The App Server component of Google App Engine:

• Hosts application code
• Handles concurrent requests
• Enforces isolation for app safety
• Maintains statelessness
• Supports Python & Java runtimes (and in fact any language than runs on JVM, including Scala)

It also supports the following services & APIs

• Memcache – Distributed, very fast, in-memory cache
• Datastore
• Uses Bigtable
• Distributed, partitioned data stored
• Arbitrary Horizontal Scaling to ‘Internet Scale’
• Replicated and Fault Tolerant
• Predictable query performance with no deadlocks
• URL Fetch – Simple Http communication with GET/POST to external services; Allows integration with 3rd party REST APIs
• Mail – Inbound & Outbound mail
• XMPP – Instant messaging
• Task Queue – Background & scheduled computation and processing infrastructure
• Images
• Blobstore – Upload and distribute large files
• User Services – User account support including federated login (via Google accounts)

Open Source Implementation – TyphoonAE

The presentation finished with a code demo, but the presenter also pointed out that there is an open source implementation of Google App Engine called TyphoonAE.

Tags: cloud, google, googleAppEngine, oscon