In the first part of this blog post, we outlined the problem of rapid development setup as well as the approaches that we have used at eBay to address the problem. We will now describe how we set up projects in the Eclipse IDE, which is well adopted at eBay for Java applications.
Let’s start by looking at two Eclipse project types – source and binary – before we describe dynamic projects, which convert between source and binary.
Source project
A source project is a standard Eclipse project created by clicking File -> New Project. A source project includes these characteristics:
- Source elements
- Dependencies to other projects or libraries (binary bundles)
- Builders that convert the source elements to binaries
With source projects, an SCM system typically provisions the source elements. Project metadata is created and optional pre-build steps are executed before compiling the source elements. The most time-consuming steps tend to be source downloads and compilation.
Binary project
The binary project is not a common Eclipse project type. Eclipse does not have a wizard to create a binary project, but it does provide this option in cases where an existing plugin or feature is imported into the workspace. A binary project has these characteristics:
- No source elements; instead, libraries with attached source
- Dependencies to other projects or libraries
- No-op builders and no source authorship
In most other ways, binary projects share the capabilities of source projects, including the ability to browse, debug, and execute code. As there are no source elements to provision and compile, the setup of binary projects is generally much quicker compared to setup of source projects.
Dynamic project (project level)
The advantage of a binary project is that there is no source to download or compile, as we consume compiled source; the obvious disadvantage is that it doesn’t provide the source authorship capability that is available with source projects. We therefore came up with a third type of project — what we call a dynamic project — which can convert a binary project into a source project and vice versa.
Let’s look at the workflow for a project-level conversion. The developer first sets up the environment with binary projects to browse, run, and debug the application. When ready to make changes to a project by editing the source, the developer simply selects the appropriate binary bundle and converts it to source.
The system fetches the source elements, compiles them, and brings them as a source project into the workspace. The system also adjusts dependencies, so that all other projects that were depending on the binary project now depend on the source project. Conversion from source to binary is accomplished in a reverse manner.
Dynamic project (source level)
At the project level, the dynamic project acts as either a binary or a source project. When we have a limited number of source elements, project-level conversion from binary to source works fine. In contrast, when a project contains a large number of elements but only a small set of those elements require authorship, conversion of individual source elements is useful. In this case, the project becomes a hybrid: it is neither a binary nor a source project. Only the source elements that need to be modified are converted from binary to source.
When the first source elements are converted, a source project is created. It contains only the converted source elements, with a dependency to the binary bundle. Due to specific tooling for dynamic projects, Eclipse exports the compiled .class elements before the libraries and binary bundles. The class loader picks the first matching class, which is the one from the source element in the workspace. In this manner, any source element modifications get reflected in the application.
Summary
Quick developer setup is very desirable yet challenging in environments with a large code base and a high velocity of changes. In web applications affording the flexibility of frequent rollouts, the number of feature releases is usually large (hundreds per month in eBay’s case), while the number of changes per feature is often limited (for eBay, hundreds of source elements).
Processes and tools help provide an environment that supports quick developer setup. We have described our journey in providing such an environment through the use of binary bundles, hierarchical repositories, and a combination of incremental source changes with binary bundles. We have described the challenges that these approaches pose as well as the approaches and ideas we have developed to overcome them.