• JDK7 for MacOS X Developer Preview announced by Hasan Rizvi (SVP @ Oracle). Available here.
• Java FX to be open sourced, and it will be proposed to the JCP (Confirmed, see press release)
• JDK8 will be released in the summer of 2013 (not 2012 as previously discussed), as announced by Adam Messinger from Oracle

There was also mention of an Oracle Java Magazine, which I confess I had never heard of before.

For a summary of this morning’s announcements, see here.

Overall, not the most exciting keynote I have seen. It opened with a presentation from Juniper Networks. I didn’t find the topic of “Programming the network” to bring networks and apps closer together particular relevant for me personally. They then rolled out a bunch of other corporate folks from the likes of Intel, Redhat, IBM and ARM. The Twitter guy did announce that Twitter are joining OpenJDK as well as the JCP though.

On to the sessions…

Some of the themes of this morning’s keynote talk were JavaFX 2.0, what will be coming in Java 8 and a new Oracle NoSQL Database. There are some links here, here and here (and I believe the videos will be posted soon here).

I just attended a very useful Java Web Framework comparison talk that I will try to post about shortly. I also met up with some of the DZone team. For now, I am going to some JavaFX talks.

If any of you are attending, let me know!

Intro

WebSockets is a technology that enables bidirectional communication between web browsers and server side processes. It provides a persistent connection between client & Server, hence enabling ‘push’ abilities where you can push data/notification to browsers from the client. Other client communication options to compare it with would be Ajax
and Comet (basically Ajax with long polling).

Specifications

• WebSockets API – This focuses on the client side programming model.
• WebSockets protocol – Describes the client/server communication in more detail. More complicated and 51 pages long. The WebSockets protocol is still actively being changed.

Browser Support

• Google Chrome 4.0.249.0 and higher (v5 has WebSockets enabled by default)
• Safari 5.0
• FireFox 4.0 beta 1
• IE 9: TBD

There was a suggestion that Apple have decided to disable WebSockets in iOS4 but this is unconfirmed and the speaker suspects Apple will support WebSockets in the future.

Server side support

• Apache Server – not supported yet
• Py WebSocket – designed for use with Apache Server (actually implemented by Google Chromium team.)
• There is also a DJango solution: django-websocket
• Jetty WebSocket Server – Jetty has supported WebSockets since last September. This seems to be a good option.

However, note that there is no standard service side Java API. Each Java container has it’s own API. Therefore it will likely be difficult to write WebSockets for one app server and do a straight port to another app server.

Projects

Other Java Projects supporting WebSockets:

• GlassFish/Grizzly (see a DZone posting on it here)
• jWebSocket
• JBoss Netty (see patch here)
• Caucho Resin

Sean finished by showing the Google Quake demo (also reported here).

The conclusion of the talk was on how to gracefully handle browsers that don’t support WebSockets (e.g. IE8). The main option here seems to be “The Graceful WebSocket“. This resorts back to using AJAX long pulling if WebSockets isn’t supported in the browser.

My Thoughts

WebSockets looks like one of the most interesting technologies that fall under the HTML5 umbrella. If it gets wide browser support (e.g. IE9), and perhaps some standardization on the server side, it could really take off and even potentially over take Ajax. Definitely one to watch…

All my notes are below…

Database Scalability Patterns

The first talk on “Database Scalability Patterns”
was by Robert Treat from OmniTI.

Database scalability patterns are part design patterns and part application life cycle.

Phases

Typically, most databases go through the following phases:
1. MyFirstDatabase
2. Vertical partitioning
3. Vertical scaling
4. Read slaves
5. Horizontal partitioning

1. MyFirstDatabase

MyFirstDatabase is just a term to refer to a simple, unsophisticated database setup.

2. Vertical Partitioning

You now have much more data and more transactions.
The Wikipedia definition of Vertical partitioning is creating tables with fewer columns and using additional tables to store the remaining columns.

3. Vertical Scaling

Adding more RAM, more disks. Basically, adding more hardware.
This is not necessarily a one time deal as you can do multiple iterations.

4. Read Slaves

“Read Slaves” or ‘Master – Slave”
I think the idea here is that you have a single destination for database writes, but multiple sources for database reads. These ‘slaves’ read database are replicated form the single write source.
This approach however does not work for write bottle necks! It only works for read heavy loads.
Typically
- full copy of data on each node
- Asynchronous
Consider
- Partial copy
- Synchronous
- Perhaps don’t use a RDBMS

Also note that the Master/Slave setup typically requires changes to your application code. But overall, the Master/Slave approach is fairly easy

Note that scaling large number of database writes can be much more difficult. One approach is to use multiple Masters, but while there are many ways to implement multiple masters, there are few that really work in a production environment! You may be able to reduce CPU, but it is difficult to reduce the I/O. Really, the master slave approach is is a fail-over solution, not a scalability solution.

5. Horizontal partitioning

According to Wikipedia, Horizontal partitioning is putting different rows into different tables. It is sometimes referred to as Sharding.
- Move each piece to own server
- Duplicate some data as needed
- When splitting the data, you must separate dependencies in the app code first!
Note that each each node is a new instance of vertical scaling

“Universal truths” of scaling databases

1) Vertical scalability is helpful for every pattern
Even in a horizontally scaled, fully distributed database, the number of nodes needed is affected by vertical scalability
2) New nodes are never free
-Adds points of failure
-Add management costs
-Add complexity to architecture
-Add complexity to your app code

Tips

- Plan for layered data sources
- Read/write connections in code i.e. have separate read vs write connections in the code to start with. Then you can have planned outages where you can still read, but not do updates.
- Use schemas to separate services (think about what pieces of data need to talk/be aware of each other, and what do not)

Sharding

The second database scaling talk was on “Sharding for the Masses” by Giuseppe Maxia (MySQL @ Oracle)

He simply described sharding as “breaking a database into pieces”.
It is used simply as a way of dealing with too much data & traffic.

Replication

One approach to scaling is replication.
Client sends a write to master and the reads to a load balancer.
The write master and the read load balancer distribute to slaves.
I think this is simply ’4. Read Slaves’ from the database patterns talk above. Again, the speaker pointed out that this approach doesn’t scale well when you have too many writes because the he ‘Write master’ becomes the single point of failure.

Homemade Sharding

So, another alternative is to use homemade sharding.
‘Homemade’ because your app contains logic of how/where to put data.
However there are 2 problems with this homemade approach.
The first is that your application logic can become very complex.
The second is that if the sharding logic is in your application, the approach doesn’t work if data is accessed via a different application!

Sharding via Spider for MySQL

So, instead of using a homemade approach, the speaker suggested using Spider for MySQL.

Google App Engine is a way to run your applications on Google infrastructure. You push your code to App Engine and it gets scaled out depending on how many instances you need.

My notes from the talk are below, but you can find the full slides here or here.

LAMP Setups

The motivation for using Google App Engine is due to the limitations or constraints of a typical LAMP (Linux, Apache, MySQL and Perl/PHP/Python) setup.

1) Single Server

A single server running the LAMP stack is not scalable and has a single point of failure.

2) Separate web & database server

The next step would be to move the Apache server and the database to separate servers. You may get better performance but this is still not a highly scalable solution and you now have two single points of failure.

3) Multiple web servers

You can provide a more scalable solution by adding multiple Apache web servers, but now you need load balancing and the database remains as a single point of failure.
In order to utilize the multiple web servers, you can use round robin load balancing. Register the web server IPs with DNS; DNS record is cached with a TTL (Time to Live). But the TTL takes time to propagate down and may not be respected, so if a web server goes does, you can still have problems. Additionally, the database remains a single point of failure.

4) Master Slave Database

The next approach is to use a Master Slave database. Note that this approach was also discussed at the Database Scalability Patterns talk, and the Database Sharding talk. This Master Slave database approach improves read throughput, but the Master is now a single point of failure for writes and may die before replication takes place.

5) Partitioned Database

So, you can migrate to a partitioned database where you will (hopefully) have better read and write throughput, but where you now have more machines and more management and you may need to update your application logic, queries and schema.

Google App Engine to the rescue

So, in order to avoid all the hassle of above, Google propose using the Google App Engine.

Cloud Computing landscape

App Engine fits into the Cloud Computing landscape as follows:
Software as a Service: Google Docs, saleforce.com
Platform as a Service: Google App Engine, force.com
Infrastructure as a Service: Amazon Web Services, Rackspace, VMWare

App Server

The App Server component of Google App Engine:

• Hosts application code
• Handles concurrent requests
• Enforces isolation for app safety
• Maintains statelessness
• Supports Python & Java runtimes (and in fact any language than runs on JVM, including Scala)

It also supports the following services & APIs

• Memcache – Distributed, very fast, in-memory cache
• Datastore
• Uses Bigtable
• Distributed, partitioned data stored
• Arbitrary Horizontal Scaling to ‘Internet Scale’
• Replicated and Fault Tolerant
• Predictable query performance with no deadlocks
• URL Fetch – Simple Http communication with GET/POST to external services; Allows integration with 3rd party REST APIs
• Mail – Inbound & Outbound mail
• XMPP – Instant messaging
• Task Queue – Background & scheduled computation and processing infrastructure
• Images
• Blobstore – Upload and distribute large files
• User Services – User account support including federated login (via Google accounts)

Open Source Implementation – TyphoonAE

The presentation finished with a code demo, but the presenter also pointed out that there is an open source implementation of Google App Engine called TyphoonAE.

What is an engineering leader?

• Serves the team
• Eliminates roadblocks
• Provides advice, guidance and helps get job done
• Mutual respect/Mutual trust
• Promotes technical and social health
• Leadership is not a waste of time…
• Being a team leader is a way of scaling yourself

The speakers broke their talk on how to be a good leader down in to two sections, which were (in good coding tradition) patterns and anti-patterns…

Anti patterns for leadership

1. Be everyone’s friend
Don’t try to be everyone’s friend because you still need to be a hard-ass sometimes. Instead, just try to maintain a healthy, working relationships. For example, try having lunch with your team regularly.

2. Treat reports as children
Give responsibility and trust.

3. Micromanage
Micromanagement breeds co-dependency. If you are off for a day, the team can’t work. They should work on their own initiative, without your commands.

4. Hire pushovers
Don’t hire people that are dumber than you! If they are smarter, you will build and better team and learn more from them.

5. Compromise when hiring
Don’t just just hire the top 10% of candidates you interview – they might not be good enough.

6. Ignore low performers
Do not ignore them! Low performers are the obstacle in the road. They will not go away. Try to motivate them. Maybe need to more-micro-manage. Have an ‘acceptable’ level bar.

7. Ignore human problems
Don’t assume the team has no life outside of work. Keep morale up.

Patterns

1. Be honest
There is no such thing as a temporary lapse of integrity!
Deliver criticism directly, constructively and with empathy.
Be honest about your motivation for making decisions.

2. Be a wise zen master
If someone asks hard questions, you can respond with questions.
Don’t try to manage people!

3. Lose the ego
You don’t need to be the smartest person in the room. Trust others and respect their abilities.

Appreciate any inquiries the team has; Accept feedback and don’t be scared to apologize. Humility != Doormat

To influence, be influenceable.

4. Get your hands dirty
Lead by example and understand the problems the team is going through.

5. Delegate
Delegate even if another team member can’t do a job as well (or as quickly) as you think you can.

6. Seek to replace yourself

7. Make waves
When you lease want to make waves, this is probably the time you should be making most waves.

8. Shield your teams from bureaucracy

9. Succeed and fail as a team
Public praise, private criticism as everyone already knows when someone messed up.

10. Be a catalyst, a teacher and a mentor

12. Set clear goals
Make sure everyone agrees on the milestones and the plan for a projects.
Otherwise you get feature creep and get side tracked.

13. Track happiness & careers
If someone is not happy,find out why.
What do team members want to be doing?
Ask them what they need?
Get everyone to work on fun stuff & dull stuff. Change it up.

Give Autonomy & Trust
e.g. the 20% time @ Google

Mastery
People want to grow; Always want to master new things; Give people a sense of purpose

Managing your manager
Act like a grown up. Encourage people to take responsibility

Pursue responsibility
Do more than is required. Deliver 110%
Take calculate risks

If you fail – learn. And document the process and what was learned.

Communicate!!

Point out obstacles

Argue
Good leaders don’t want yes men
Give back structured criticism

Conclusion

3 words:

• Serve
• Respect (give more, get more)
• Motivate

Stop being a manager, be a leader!

Finally, the speaker recommended checking out a talk by Dan Pink on Intrinsic vs Extrinsic Motivations. I think this is it.

As well as the Scala introduction classes, there were interesting talks on Akka, Simple Build Tool and Specs….

Akka

I attended a lecture by Jonas Boner on the Akka library. Jonas believes that it is simply too hard to write correct concurrent, scalable applications and that the threads and locks approach to concurrency taken in Java is far from the ideal approach. So, he created the Akka project to provide a better solution.

Akka is an opensource library ,with a Scala and Java API, whose MO is “Simpler Scalability, Fault-Tolerance, Concurrency & Remoting”. It achieves this through the use of the Actor Model. As well as actors, Akka provides Fault tolerance (inspired by the approach used in Erland) using a Let It Crash approach. Akka also makes use of Software Transactional Memory.

It seems like a well written project with a lot of good ideas. I would like to learn more about it (and perhaps hear of it being used in some enterprise apps) before I would attempt to use it in anger, but definitely one to keep an eye on…

Simple Build Tool

The second interesting talk was by Mark Harrah on SBT (Simple Build Tool).

SBT is for building projects. It is all about Scala: written in Scala; builds scala; defined in Scala.
It uses Ivy for dependency management, can be run in interactive mode or via triggered execution (~) to run when the source code changes.
The tool seemed neat and effective when he demoed it and this was reinforced by the other Scala presenters giving it a very big thumbs up. If I ever get a chance to start developing Scala in anger, this will be the first tool I would look at…

Specs

The final talk of interest was on Specs, by Eric Torreborre.

Specs is a is a Behaviour-Driven-Design framework which provides a simple and typed language to create specifications. It seemed to be a way to create specifications as tests (with integration with JUnit and JMock) in a way that is understandable by both business and technical folks alike, in a DSL like manner.

Part2: Practice

10 ways to improve your code

1. Composed Method

The concept of ‘Composed Method‘ Comes from Smalltalk Best Practices Patterns.
The concept involves dividing your programs into methods that perform one identifiable task. If you find yourself chunking a method (using blank lines) you should probably be refactoring

Benefits of composed methods

• Shorter methods are easier to test
• Method names become documentation
• Large number of cohesive methods
• Discover reusable assets

2. Test Driven Development

(And Test Driven Design!)
TDD is the first consumer of your code – it makes you think abut how the rest of the world will use it

• Creates consumption awareness
• Forces mocking of dependent objects
• Naturally creates composed method

Extroverted objects (e.g. where you create dependent objects in theconstructor) ‘reach out’ to create other objects, i.e. adhoc object creation
Introverted objects move object construction to a few simple places. Dependency Injection is a way to achieve introverted objects.

3. Static analysis

• Findbugs – byte code analysis
• PMD – Scans Java source code and looks for potential problems
• CPD – Copy Paste Detector

4. Good citizenship

a) getters & setter != encapsulation

If user code has getter & setters, it can break encapsulation too!
Create atomic mutators for dependent fields
e.g. Address should have a single setAddress method, not setCity & setState

Instead, only have getters & setter when you need them for other methods.
And you shouldn’t have to test getters & setters – will be tested through other code.

b) Constructors

Specific contract for how to create valid objects
Q: How often is a blank object valid?
A: Never!
Therefore, don’t provide default constructor for domain objects
Push back on frameworks that require default constructors

c) Static methods

One good use for static methods is completely black box use. No state involved i.e. for stateless, stand alone methods.
But if you mix static & state, you have problems.
e.g. Singletons!
Singletons are bad because…
1) Mix responsibility (creation and logic)
2) Untestable
3) The object version of global variable

Avoid using Singletons and instead
1) Create a plain class
2) create a factory to create POJO of that class

5. Yagni

You Aint Gonna Need It!

• Discourages gold plating
• Don’t indulge in speculative development – as that increases software entropy
• Leads to frameworks, in a very bad sense e.g. Avalon is framework to build frameworks

Frameworks extracted from working code can be good.
Frameworks created as speculative development are usually bad.

6. Question Authority

Test names are special
Use underscores instead of Camel case (only for test method names though!)
Much easier to read

APIS
Use fluent interfaces
Makes code readable to non-developers
But this vialoates the JavaBean spec
(need default constructor, all set methods must return void)
Therefore can’t use JavaBeans spec for fluent interfaces

Summary – Know when to break the rules

7. Slap

Single Layer of Abstraction Principle
Keep all lines of code in a method at the same level of abstraction
Jumping abstraction layers makes code hard to understand
(See a blog posting on this here)

8. Polyglot programming

Consider leveraging existing platforms with languages targeted at specific problems and applications. In other words, use the right tool to get the job done.
e.g. Java is bad at threading, consider Jaskell or Scala
e.g. If there is schedule pressure, consider JRuby on Rails, or Grails
e.g. For every day tasks done quickly, consider Groovy or Ruby. or Swiby (Ruby / Swing)

“Stop banging rocks together & get some work done!”

9. Learn every Nuance

Learn all the quirks
For example, consider:

• Using Reflection (no longer slower). Can give elegant solutions to problems.
• Regular expressions

Learn the nuances of your tools, then tell the other people on your project.

10. Anti Objects

An antiobject is a kind of object that appears to essentially do the opposite of what we generally think the object should be doing.
Consider using them if they bring elegant solutions.

The talk was in 2 parts: Mechanics and Practice.
Below are my notes from the first part of the talk (see here for part2), but you can also get the original slides form here. The talk is based on his book of the same name, The Productive Programmer. Which is similar in theme but not to be confused with the “The Pragmatic Programmer” book.

Part1: Mechanics

1. Acceleration
2. Focus
4. Canonicality (applying the DRY principle)
3. Automation

1. Acceleration

The basic premise of ‘acceleration’ is that typing is faster than navigation and that tools, especially for repetitive tasks, are your friend.

Operating System accelerators

For o/s accelerators, he pointed out several useful tools:

• Windows: explorer address bar (alt-d shifts focus to the address bar)
• Apple: In Finder, use apple-shift-g to go to Folder; There are also a bunch of other shortcuts like apple-shift-h = go to home directory
• Firefox: Use the numberfox plugin – I looked in to this. I think it is this plugin, but unfortunately, it is only available for Firefox 1.5-2.0
• He also suggested using smart help on Mac Leopard, but I’m not sure what this refers to…

Clipboards

Neal pointed out that having only 1 clipboard with 1 entry is very limiting and suggested these tools to help:

• Windows: clcl (infinite/100 clipboard history)
• Mac: iClip ($)
• Mac: Jumpcut

Directory Switching

‘There and back’ for directory switching

• pushd – pushes a directory on the stack
• popd – pops it back off

(Note that these tools are also available on Windows)

Command Prompts

Graphical explorers better for some things, but the command line is better for others. The following are some tools to easy command prompt use:

• Windows: ‘Command prompt here’ power toy [Update: This is now available by default in Windows 7. By holding shift and right-clicking a folder, you'll see additional commands not listed on the standard menu. See here for more details]
• Windows: Command prompt explorer bar – This looks very cool!
• Cygwin: bash here
• Mac: Path finder ($)

Prefer Keyboard to Mouse

When coding, always prefer keyboard to mouse

Learning shortcuts

• Make yourself use the shortcut, even if you’ve found it via the menu!
• Have someone else pester you about it – e.g. pair programmer
• There is an app called KeyCaster than displays what keys were pressed
• Key promoter plug in for IntelliJ
• mousefeed for eclipse
• Try turning off menu options! Forces you to use keyboard shortcuts
• Use flash cards & cheat sheets

Java IDE Examples

All of our hierarchies are too deep, including in the file system & in code.
He suggested the following tools for IntelliJ:

• goto class (like ‘Open resource’ in Eclipse?)
• Can also open based on capital letter patterns
• goto symbol (opens a class based on a symbol (e.g. var name) in it
• introduce variable (does left hand declaration for you)
• introduce variable redux
• escalating selection

Templates

All major IDE’s and coding text editors support templates in some form.

• Use templates, for example in IntelliJ or Eclipse for things like parameter substitution, default values, repeating values
• Learn the language of your templating engine e.g. Velocity in IntelliJ
• Every time you type something for the 3rd time, templatize it!

Key macro tools

Key macro tools are like templates, but at the operating system level. Use them to avoid typing the same commands over and over again.
Windows: AutoHot key
Mac: TextTxpander ($)

Focus

Try to avoid distractions by doing simple things like:

• Get a comfortable chair
• Use dual monitors (google dual monitors and productivity to see proof!)
• Have monitors sitting immediately in front of you
• Use a good keyboard & mouse
• Don’t get carpal tunnel! (http://en.wikipedia.org/wiki/Carpal_tunnel_syndrome)
• Try moving mouse to your ‘bad’ side, or use on left AND right side (Using the mouse on your ‘bad’ will force you to use keyboard more and will also make you ambidextrous!)
• Motivate yourself to learn keyboard shortcuts
• Have admin privileges for the O/S (If you are not given permission, start a spreadsheet of all the time you waste because you can’t do what you need to due to lack of admin rights; Show your boss how much time it is wasting)

In general, offices are terrible environments (too many distractions) – War rooms work better.

Locus of attention

Anything that happens outside your locus of attention breaks flow.
Flow is when you have total concentration and time disappears (‘in the zone’).
It is hard to get in to and easy to break (like sleep in that respect).

To avoid breaking flow/concentration, try:

• Tweak UI – A tool that allows you to easily manipulate your Windows environment. For example, you can turn of balloon tips for ever.
• Screen dimmers – Allow you to automatically set you background to dark after a set time
  • Windows: Jedi Concentrate
  • Mac: Doodim
• Internet blockers
  • Mac: Concentrate – Have to reboot machine to unblock. Can configure to only block certain sites.
  • Windows: Jedi Concentrate

The higher the level of concentration, the denser the ideas, so to maintain concentration:

• Turn of notications
• Don’t keep email open – email is best done in batches (e.g. deal with at 9, noon and 3)
• Put on headphones
• Create office “quiet time”

Focus techniques

Search beats navigation

Search is (increasingly) better than navigation, so try to replace file hierarchy navigation with search

• Desktop search. e.g. Google Desktop search(aka quick search box)
• Larry’s “any text file” indexer

Rooted Views

A rooted view is a specialized explorer view with a base folder that is ‘rooted’ at the folder you specify.
(on Mac, this is simply the directory shortcuts on the left hand side)

Use virtual desktops

• virtual desktop manager power toy
• VirtualWin (better?)
• Spaces (built into leopard)

Canonicality

DRY – Don’t repeat yourself

ORM

A database schema can be made up of DDL + xml + pojo
Choose 1 and generate the others from that
e.g. generate xml & Pojos classes from schema
(If you need to add to a class’s logic, make the generated code Abstract, and the class that extends it is where the extra, custom code gets added)
Or, generate the schema & POJOs from Hibernate mapping files

DRY Documentation

e.g. generating wiki pages from subversion check in details (see SVN2Wiki)
DRY Diagrams – Generate based on code
DRY Schemas – Schemaspy

Automation

In general, don’t do by hand what you can automate!

• You should have a one command build (if, on a brand new machine, you can’t issue one command to get build, and another single command to build, then you have a flawed build process)
• Continuous Integration (good for building, metrics reporting)
• Version Control (Essential!) – Do not comment out code! Remove and use VCS.

Subverting other tools

Selenium (open source tool for UAT)
Includes a side project called selenium IDE
Side use: Allows you to automate debugging wizard style web apps! (you need to test one page, but you need to navigate through 10 pages to get to it – use Selenium)
Record your interaction the FIRST time you walk through the page
Cuts debugging time!
Selenium defines an interaction API for web apps
Have your Q/A dept record bug discoveries

You almost never do anything just once

• Work like a craftsman, not a laborer
• Build shims & jigs
• Building a tool takes longer, but you are building assets
• Develop Bash-fu

Why Automate?

• Time sent automating can take multiple times longer but can be used multiple+ times. And can accidentally can become a major part of project!
• Allows throw-aways to grow into assets
• Allows unit testing, refactoring, support
• Solving problems by hand makes you dumber!
• Steals concentration
• Squanders focus

Justifying automating

• Timebox! – Set a reasonable amount of time to POC
• Evaluate @ end of timebox
• Abandon if it is not working out

Analyze the ROI

• How long does it take to do X times?
• What are the consequences of doing it wrong?
• Automation is about time saving and risk mitigation

Last and by no means least – Don’t shave yaks!