Go beyond Java with CI server and Artifactory

During the last couple of years, continuous integration (CI) and automated release management methodologies have become much stronger in non-Java builds.

Number of familiar tools are used for these methodologies, like the version control system to manages your sources, your build tool to actually build your software from sources and  your build server, which builds your software continuously using the build tool. But is there something else missing? Let's look at a simple diagram, which describes your CI process:

1. Developers fetch needed dependencies (3rd party libraries, other modules).
2. Developers write new code.
3. Code is committed to VCS.
4. Build server polls VCS for changes.
5. Once discovered, the build server builds the software: compiles it, runs tests, and assembles artifacts.
6. Built artifacts are published to QA, staging, or even directly to the end users.


As you can see, everything except stages 1 and 6 is well known and familiar. But what about those two stages? You need a dependency management mechanism for them.

In this post, we’ll illustrate the options for automated builds with dependency management in non-Java builds or when not using .NET with NuGet (use it, if you can! Here’s our take about it). The proposed solution works for any kind of build, be it C, C++, C#, iOS and Objective C, Python, or whatever.

So, let’s see how you can implement CI with dependency management for non-Java projects. Here are the options with their pros and cons:

#	Solution	Pros	Cons
1	Dependencies in VCS	- Use proven build tools - Simple setup – you use VCS anyhow	- VCS won’t fit for binary dependencies. You can read why here.
2	Build tools with built-in dependency management like Maven and Gradle	- Build tool with dependency management	- New tool to learn. Both Maven 2 and Gradle have a pretty steep learning curve. - “Adapted” from Java. While there are ways to build “native” applications with them (easier with Gradle, harder with Maven) it still feels, let’s say, artificial, imposing their new conventions.
3	Declare dependencies on builds in your build server	- Use proven build tools	- Only works for inter-project dependencies (which were built by the same build server) - Not flexible enough       - Includes/excludes       - Layout changes
4	Use shared dependencies storage, a.k.a. “repository” (FTP, file server, etc.)	- Use proven build tools	- Manual repository populating - Managing the repository

As you can see, none of the solutions are without the cons.

Here comes Artifactory

Artifactory is  a binary repository – built with binaries management in mind in terms of versioning, management, security, and build servers integration.

As such, using Artifactory completely eliminates the cons of the fourth solution. Your build can populate the repository using powerful REST API. Management is easy and streamlined.

But there’s more. By using Artifactory CI integration (for Jenkins, TeamCity and Bamboo), you can also avoid the cons of the third solution listed above. First, you can specify which artifacts you are willing to publish. They will be uploaded to Artifactory in the end of the build. Second, you can specify which dependencies your build needs. They will be downloaded from Artifactory during the run. All this simply by using build server UI.

The Artifactory Jenkins plugin, for example, defines as part of the Generic build support, a simple pattern-based language that allows you to express what artifacts from the build are going to be deployed to the repository. This includes attaching dynamic searchable properties to these artifacts upon deployment.

First, enable Generic-Artifactory Integration in the Build Environment:

Next, configure your custom deployment and resolution rules. Let's start with deployment (publishing):

In the above example, we configured the following rules for deployment. All the artifacts will be deployed from the working directory into the ‘libs-release-local' repository (configured above), maintaining the path for each file:

Pattern	Meaning
**/x64/*.dll=>x64Win	Deploys all DLLs to the 'x64Win' directory
**/*.zip=>winFiles	Deploys all zip files to the winFiles directory
unix/*.tgz	Deploys all tgz files under the unix directory to the root directory of the target repository

The plugin also controls which artifacts to resolve from the repository before the build starts. The resolution allows you to specify smart search patterns:

In the above example, we configured the following rules for dependencies resolution:

Pattern	Meaning
libs-release-local:x64Wi/*; compatibilityLevel =medium,high	Resolves the files from the x64Wi directory of the libs-release-local repository to the root of the workspace, but only if the ‘compatibilityLevel' property is set to be above medium
libs-snapshot-local:*.zip=>winFiles	Resolves all zip files from libs-snapshot-local repository to winFiles directory under the root of the workspace
libs-snapshot-local:unix/distro.tgz=>linuxFiles	Resolves the distro.tgz file from unix directory in libs-snapshot-local to linuxFiles directory under the root of the workspace
libs-release-local:**/* @winx64_build#released	This example shows dependency to artifacts produced during an earlier build that has been marked with a "released" status

From a configuration standpoint, this support for automatic dependency declaration and publishing makes it very easy to support a release flow where multiple artifacts are collected towards building the final release.

You are more than welcome to give it a try. The publishing functionality described above is available in the free open source version of Artifactory. If you want to configure dependencies resolution, go ahead and download the Pro version evaluation.

Dependency management is critical for continuous integration and rapid release. By adding Artifactory to your CI stack, you can easily overcome the lack of such functionality in your build tool, leaving the dependency resolution and artifacts deployment to the binary repository and your build server.