Supporting multiple platforms – sharing our experience

May 6, 2013 by Nikita Salnikov-Tarnovski

This post is part of our open culture. We do not believe into closed processes, so we decided to share some interesting concepts about our development infrastructure. Or more precisely – how do we support multiple platforms in Plumbr development.

Lets start with our goals in mind. Among other goals we are aiming towards, the one adding vasts amounts of complexity is the number of platforms we need to support. If you have thought that for Java it will all be about Write Once, Run Anywhere then – make a second guess.

It all falls apart the very same minute you create something truly low-level and optimized to the last possible bit. Suddenly all kind of unexpected small details start to matter, such as the garbage collector selection, minor differences in the classloader implementations or the way JVM vendors have decided to interpret certain parts of the JVM specification.

Now having to support this low-level software running in all kind of unimaginable environments you have set up a strong foundation to failure:

  • Operating system: Windows, Linux, Mac, Solaris x86, Solaris SPARC.
  • Underlying architecture: 32-bit or 64-bit.
  • JVM vendor: Oracle HotSpot, Oracle jRockit, IBM JDK
  • Java version: 5, 6, 7 and 8.
  • Application Server: Tomcat, Jetty, JBoss, Weblogic, Websphere, etc.

Most of the Java developers are lucky and have never had to dig into the hairy details of supporting multiple platforms. After all, most of the work being done by the Java developers is related to enterprise applications being deployed to a single production environment. Even if the development / staging environments differ from the production, it is usually solved in ad-hoc manner of figuring out whether to use forward or backward slashes in a particular file system.

But as we were not so lucky as we needed to support as many different environments as possible. And as our early releases demonstrated, it is nowhere near OK to test the code just on our Mac OS X development environments nor in our original Ubuntu testing environment. The bugs just kept surfacing.

So we had to figure out what to do with the aforementioned configuration matrix. Which is actually only a subset of what will be needed eventually, as we have not differentiated between the Windows releases nor different flavours of Linux nor the minor JVM releases. But even if we multiply the different configurations in the far-from-complete set of platforms listed above we reach to the 600 different configurations we have to support. To give you an idea about the configurations:

32-bit Windows XP with Oracle Hotspot 1.5.0_10 JVM running a Weblogic 8 application server

versus

64-bit Ubuntu 12.10 with Oracle jRockit running an Apache Tomcat 6.0.23 application server

From here it is relatively easy to fill out the remaining 598 configurations. But somewhere along this boring process you will start discovering close-to-impossible combinations, such as

  • IBM Java running Oracle application servers.
  • Oracle jRockit running IBM Websphere
  • SPARC boxes with Java 7 and above

After understanding this, our original goal of “supporting as many possible configurations as possible” was quickly replaced by “support as many users as possible”. Digging into the statistics submitted by our users also gave confidence in doing so – the truly exotic combinations are close to non-existent.

So you can reduce the size of the matrix somewhat. In our case we were able to cut the required infrastructure to more than half of its theoretical size. But it still left us with more than 200 environments to support. Which, as you can imagine is still way too much for a human being to handle without some clever tricks and techniques in your belt.

But this will be the subject for the next post covering our internal goals and implementation details about:

  • Automated infrastructure. Build infrastructure has to be able to launch and destroy different configurations automatically.
  • Automated tests. Tests run on different environments have to be automated
  • Support multiple versions. We need to support older releases independent of the ongoing future development
  • Backward traceability. When receiving an error report from an earlier version we need to be able to link it with the exact version.
  • Obfuscation map compliancy. As we use code obfuscation we need to be able to use the proper obfuscation map for proper version.

If you wish to be alerted about our next posts then subscribe to our Twitter feed

The post might seem familiar to those of of you who were participating in JavaOne Moscow this year. Indeed, you had a chance to hear me on stage with the same presentation. But the rest of the ~8,310,700 who did not have a chance to be present I hope I was able to introduce some interesting concepts.

Can't figure out what causes your OutOfMemoryError? Read more

ADD COMMENT

COMMENTS

Huh? You’re writing a low-level memory analyzer for multiple JVMs. Most developers aren’t doing that and therefore do not have even remotely the same number of issues as you would. I build for mac, linux and windows all the time without hardly a glitch. Dude, you’re the exception – not the rule.

Dan Howard

Surely this is not a problem with modern memory chips. I heard that older chips in early model PCs had leaking problem, but after so many years of PC working good I find it hard to believe this about leaking memory.

Weng Fu

Weng Fu, your post here [1] was funny. Now it’s starting to get annoying. You are in the way to become a troll, but you still have a chance. Don’t wait till it’s too late.

[1]https://plumbr.eu/blog/how-to-create-a-memory-leak

Alied Perez

Can't figure out what causes your OutOfMemoryError? Read more

Latest
Recommended
You cannot predict the way you die
When debugging a situation where systems are failing due to the lack of resources, you can no longer count on anything. Seemingly unrelated changes can trigger completely different messages and control flows within the JVM. Read more
Tuning GC - it does not have to be that hard
Solving GC pauses is a complex task. If you do not believe our words, check out the recent LinkedIn experience in garbage collection optimization. It is a complex and tedious task, so we are glad to report we have a whole lot simpler solution in mind Read more
Building a nirvana
We have invested a lot into our continuous integration / delivery infrastructure. As of now we can say that the Jenkins-orchestrated gang consisting of Ansible, Vagrant, Gradle, LiveRebel and TestNG is something an engineer can call a nirvana. Read more
Creative way to handle OutOfMemoryError
Wish to spend a day troubleshooting? Or make enemies among sysops? Registering pkill java to OutOfMemoryError events is one darn good way to achieve those goals. Read more