From the Frog's Mouth

JFrog jumps at Devops opportunity with continuous integration repository

2012-04-12T18:13:00.000+03:00

Analyst: Jay Lyman

Israel-based JFrog backs the open source Artifactory repository management software, where it finds audience with both application developers and IT operations teams.

Its positioning between these two parties managing one of their common challenges in software artifacts and binaries places JFrog at the crux of the devops trend that is pulling development and IT operations together. While its Artifactory software and repository management remain somewhat unique in the industry, we see a growing list of competing vendors and categories, particularly as more development and operations center on cloud computing.

The 451 Take

JFrog and its repository management software benefit from their unique position at the crossroads of software developers and IT operations, or dev! ops, where the two come together primarily for speed and efficiency. JFrog's early all-in bet on the Jenkins continuous integration server, a fork of the Oracle-owned Hudson, appears to be paying off as the market continues to favor Jenkins. Looking ahead, it will be critical for JFrog to differentiate itself as other vendors and categories cross more over into competition. Also key will be JFrog's ability to integrate and grow with PaaS offerings in the market, but we expect we will see the company consistently here.

Context

JFrog, based in Netanya, Israel, backs the Artifactory binary repository management software, which is open source under the LGPLv3 license. The company was founded in 2008 by its current CEO Shlomi Ben Haim, its current CTO Yoav Landman and current Chief Architect Frederic Simon, all of whom haved worked in the Java and open source communities together. JFrog has about 16 employees, most of whom are engineers, and plans to have a h! eadcount of about 40 by the end of 2012. The company launched ! its first paid product Artifactory Pro at the end of 2009, starting with some key, large customers including LinkedIn and Netflix. Today, JFrog reports almost 250 paying customers and its binary repository management software running on the same number of servers. JFrog says it is currently considering venture capital investment options, but is well funded from its own income and angel investment from its launch.

Products

JFrog's Artifactory product manages binary repositories as software moves through development to testing and quality-assurance and eventually to deployment. The software sits as a proxy between other development tools used in managing software repositories and continous integration, such as Apache Ant and Maven build automation, Ivy dependency manager and Gradle project automation. Artifactory caches software artifacts so repeated downloading is not necessary. Artifactory also blocks unwanted or external requests for internal repository artif! acts, controlling how, where and by whom artifacts are deployed. This helps to manage artifacts and third-party dependencies used by developes and IT operations, which can also share repositories among different departments or teams using Artifactory. The Artifactory software itself is built on the Java Content Repository, which helps support high concurrency and data integrity, the company says. Artifactory's underlying storage, which is backup ready, also supports the attachment of searchable XML metadata and user-defined properties via JFrog's OpenMetadata technology. The software features granular permission controls that can be managed through a simple UI.

JFrog offers its Artifactory repository management software in two versions, Pro and Cloud. Artifactory Pro is an annual subscription that includes a set of paid add-ons bundled with the open source software. The Pro version, priced at $2,350 per year, includes updates, full maintenance and bug fixes as well as a growing list of add-ons for continuous integration server support, resource management, license control, search and other capabilities that go along with the repository management. The cloud version, Artifactory Online, is the same offering in SaaS form with SLA support. Users can quickly set up private repositories using the service without the typical requirements of configuring and customizing a server and access.

The newest version of the software, Artifactory 2.6, is expected in the second quarter of 2012 and is focused on customizable release management. Much of this centers on extending Artifactory management beyond pre-staging to encompass more of the software release process. The idea is to speed the time between software testing and release by maintaining consistency and manageability of software repositories.

Technology

JFrog reports that the use and proliferation of cloud computing has resulted in an enterprise software release process that requires the management of software binaries and repositories much earlier in the effort. This coincides with the trend of devops, which demands that developers and IT operations work closer together and consider each others' roles and responsibilities more thoughtfully. JFrog says that while binaries are closely tracked through testing and QA phases, they can quickly get lost once code hits production. Thus, Artifactory i! s aimed at supporting that move of software to production. JFrog says it is gaining traction with Artifactory Online because of a move by major players, including SpringSource, to put more of their development and support in the cloud.

Artifactory also integrates with a key devops technology: continous integration servers, including Bamboo , TeamCity and Jenkins. JFrog is also benefiting from its decision to support Jenkins, which is a fork of the Hudson continuous integration server from Oracle. While we saw the market supporting both servers last year, it appears there is more momentum and vitality for Jenkins currently and this appears to be contributing to traction for JFrog.

Another part of JFrog's current growth centers on its focus on application programming interfaces (APIs), which have emerged as critical for promoting communities around connectors and plug-ins. JFrog stresses the importance of giving its users and customers a stable, documented API along with examples. JFrog also highlights that APIs are a sort of common language between software developers and IT operations teams, which both work closely with APIs to make things work.

Finally, JFrog is also adding more support for .Net and the Windows world, particularly with its recently-added support for NuGet, a .Net package manager. Through an add-on, Artifactory can support and proxy NuGet packages for Visual Studio .Net applications. This is significant as we see .Net and Windows catching up in devops after trailing to mostly Linux and open source programming frameworks and languages the previous few years.

Integrations and partnership

Artifactory is commonly deployed alongside other key devops technologies, including the continuous integration servers such as Bamboo and Jenkins as well as server automation frameworks such as Chef and Puppet. The Artifactory software also works with a range of other development tools in dependency management, build and project automation and other areas. JFrog also partners with CloudBees, a PaaS provider that is centered on the Jenkins CI server.

Competition

Its work in Java and Apach! e means that JFrog is competing mostly with Sonatype, backer of the Apache Maven and its own Nexus repository management software. While JFrog supported the Jenkins fork of the Hudson CI server last year, Sonatype remained supportive of Hudson. This continues to be the case today and while the market does seem to be favoring Jenkins, the fact that Oracle still backs Hudson may intensify the competition from Sonatype. Another category of competitors comes from the code vetting and management world, where vendors such as Black Duck continue to extend further into the development process. Other competitors here include OpenLogic, Palamida and Protecode. Another area that may represent more competition for JFrog going forward is cloud monitoring and management. For example, Electric Cloud is a cloud automation and devops vendor that includes artifact management in its offerings. JFrog must also compete with use of other open source software for repository management, such as th! e Apache Archiva repository manager.

SWOT Analysis

Strengths

JFrog and its Artifactory repository management software sit at the crux of the growing devops trend in its technology and support for both software developers and IT operations.

Weaknesses

The company remains somewhat obscure and is lesser known than other vendors in the small but growing devops space.

Opportunities

The growth of devops practices and processes among both newer, Web 2.0 type users and more mainstream enterprises could provide more than enough fuel for JFrog's growth.

Threats

Other vendors, particularly those dealing with software development and deployment in the cloud, are increasingly offering similar repository managemenet features and capabilities.

QCon 2012 - Perfect as Everything in London Should Be

2012-03-16T09:06:00.000+02:00

It was JFrog's second QCon London, and it just gets better. Imagine: even the London weather was perfect, not to mention the sessions, booth traffic, show organization and food (what, you say, good English food? Well, great IndoPak food, at least). Due to high demand by sponsors, the exhibition took place on two floors (as opposed to one floor last year). Our JFrog booth was located in the same place as in 2011. We’re getting used to the place and are looking forward to returning next March!

The speaker panel was extremely impressive, featuring gurus like Martin Fowler, Adrian Cockcroft (the man behind Netflix’ cloud), Dwight Merriman (co-creator of MongoDB), Damien Katz (creator of CouchDB) and Rich Hickey (creator of Clojure).

The conference started with two training days - six full-day tutorials each. From my perspective, the most interesting two tutorials were “Cloud Architecture” by Adrian Cockcroft, where he shared the architecture, best practices and decisions behind Netflix’ cloud (which Artifactory is proud to be a part of) and “Continuous Delivery” by ThoughtWorks’ Tom Sulston (that’s as close to the roots of the famous “Continuous Delivery” book as you can get). For me, the most fascinating thing in the Continuous Delivery process as ThoughtWorks sees it, is that its virtues are exactly the same as we based our Artifactory upon back in 2006: DevOps automation and rapid release cycle. We appreciated the validation of our concept.

The remaining three days of the week were all-day sessions. It’s impossible to review such a significant number of talks in one blog post, so I’ll concentrate only on the excellent keynote addresses (with one exception).

Martin’s conference-opening keynote speech was about data. The main feature of modern data is that it is bigger than you think. Polyglot storage in general and various kinds of NoSQL look like the right solution.

My favorite keynotes are usually the evening ones. A beer in your hand makes any amusing talk even more enjoyable. One named “Developers Have a Mental Disorder” I couldn’t miss! Greg Young gave a great show, funny and entertaining, about serious dilemmas in software development that we, the developers, prefer to ignore. The brightest example, of course is the downside of DRY (did you ever think about one?). By removing duplication, we increase coupling, which can be much worse.

Thursday morning’s keynote address was delivered by Rich Hickey. He spoke about the differences between “Simple” and “Easy”. Sounds pretty similar, but in fact they are very far from being the same. Antonyms to the rescue: simple vs. complex, while easy vs. hard. Now it’s clear - we need to strive to prevent and remove complexity (go simple) without being afraid of the hard. Choosing easiness may end up creating complexity. Things which are easy, but complex, include OOP, mutable state, imperative loops and frameworks (especially ORM). Things which are simple, but not necessarily easy (at least not until you get them), are LISP-ish languages, like Clojure.

My session also took place on Thursday, in relatively small room, about 70 people. I was more than happy to see that it was almost packed. I spoke about building trust in your build process by selecting the right tools for the job (of course, we consider Artifactory as one). I also spoke about the problems of DevOps in the word of Continuous Delivery in the cloud and the rapid release cycle of SaaS applications. I stressed the huge timeshare of binaries in ALM process and the importance of using a tool that really understands binaries to deal with them. That’s exactly the reason why we developed Artifactory.

Half of my session was dedicated to live demos, which went smoothly, incredible as it may sound. According to the feedback received, my talk was well accepted, and hopefully will be useful to some of the attendants for building easier and more reliable release processes. The Q&A session continued at our booth, where we repeatedly did live demonstrations and received excellent feedback each time. If you want to get a feeling of my talk, here’s the slide-deck.

Friday was the last day of the conference. It started hard with a highly technical keynote address by John Allspaw with a scary name: “Resilient Response In Complex Systems”. For someone like me, who doesn’t deal on a daily basis with Disaster Recovery, the session was astonishing. Looking behind the curtain of that kitchen reveals a totally different way of thinking and planning. It may be how individuals and teams have to perform during a disaster (e.g. personal heroism is bad even if successful; it sends the wrong message to the team), or simulating disasters on live production systems (I never could even dare to think about that). The most obvious, but still eye-opening advice that John gave is to learn from successes, not only from failures. It can give us a lot of information and happens much more frequently, no? The only organization with which I am familiar that embraces that technique is the Israeli Air Force.

To sum up, the conference was great by every measure: technical sessions, attendance, networking, Artifactory exposure, and after-show quality time. Thank you, InfoQ, for this wonderful event in London. QCon was a great starting shot for JFrog’s “Busy March”. You still can catch Fredric and Yoav giving talks on various events in US and Europe.

Dependency Management with .NET - Doing it Right

2012-02-06T13:29:00.000+02:00

The problem of dependency management is neither new nor original, it exists in all development platforms, and .NET is no different.
Let’s go through different solutions and see how they perform. I’ll list them here in no particular order.

Keeping dependencies in your source control
That’s a very popular solution, and for a reason. The benefits are obvious. Here are some of them:

No setup. You already have your source control in place (hm, I hope you do!). Add \bin directory, and you are fine.
No learning curve. Developers are used to work with source control.
Shared. The whole team gets changes and updates from the server as they occur.
Enterprisy (in a good way). The software is proven, backed up, DRP is done.

Sounds good, doesn’t it? So, what’s wrong with it? Only one thing - source control systems are designed to control, well, sources. As such they aren’t so great in controlling binaries. These are the shortcomings we all encountered during the years of Version Control System usage for dependencies:

It isn’t a proxy. VCS can’t download the dependency you need from a central repository when you need one. You need to manually download it and add it to VCS. The history starts from there - you just lost the link to the original file. So, you work hard, and on top of it lost information; this repeat itself for each new dependency.
Versioning mismatch. Source files are versioned by their content. VCSs know how to diff them and understand what changed. Binaries, on the other side, usually versioned by their name. From VCS point of view they are different entries, each one without any version history.
Some very popular VCSs (like Subversion) can’t obliterate files. That means - once a file was added, it stay in the repository forever. That’s not a big issue for small source files, but can become quite a pain when it comes to obsolete large binaries.
Source control knows how to search sources. And, of course, the most important type of search is by content. Searching for binaries is different: what matters there is the location, structure of the file name and, in case of archived artifact, the contents of archive.
The permissions scheme of VCSs is tailored for versioning sources (again!). For example, there is no override permission. That’s because overriding sources is something we do all the time (that’s what diff is for in VCS) - it’s the same security level as, let's say, adding a new source file. With binaries the situation is very different. While adding new binaries is fine, overriding released binary is something that shouldn’t be done, one should have a special permission for it.
Distributed VCSs, awesome by themselves, are particularly unsuited for handling big binary files. When cloning a remote repository to your machine you are bringing all the history of all the files in it. Now just think about all the huge binaries sitting there...

As you see, the conclusion is simple - we can do better. Let’s try something specialized for binaries.

GAC and WebGAC
Global Assembly Cache is, on contradictory to VCS ,tailored for storing binaries. It understands versions, prevents conflicts, and generally does a good job being your local dependencies storage. The main problem with GAC is being local, which means - each and every developer should take the binaries from somewhere and install them in their local GAC. You see the troubles coming in that setup, don’t you? WebGAC to the rescue here. It's essentially GAC shared by WebDAV and enables clients to fetch dependencies from the server, simplifying dependencies management for a team. Let’s do our pros/cons math. The benefits:

GAC is good, standard and proven solution for binaries management. It deals well with versions.
WebGAC is a central binaries repository for a team. Every team member synchronizes with it.
WebDAV is popular well-known HTTP extension with locking, security management, etc. Working with the Apache WebDAV module is generally straightforward.

Let’s see what won’t work so great with that solution:

It isn’t a proxy. WebGAC can’t download the dependency you need from a central repository when you need one. You need to manually download it, add it to WebGAC and only then it becomes available to the team. Not only must you work for every version of every dependency needed, the link to the original file is lost.
No notion of packages. GAC contains single dlls. You install them one by one. But think about NUnit, as an example. It contains about dozen of dlls along with various xml and configuration files. How can you install it to GAC?
Security is cumbersome. You’ll need to configure Apache Server’s security, and even then it won’t be flexible enough to determine between deployer (a user that can publish private dependencies) and promoter (a user that can move dependencies from a private repository to a public one).
Search is basic. WebDAV by itself only knows about files. It doesn’t care about the structure of the filename, or about the presence of Strong Names.

Looks like we still didn’t find what we are looking for, and then...

Here comes NuGet
This is something else. NuGet designed to be “a developer focused package management system for the .NET platform intent on simplifying the process of incorporating third-party libraries into a .NET application during development”. That’s exactly what we need. Let’s look how great it is:

Manages packages, not dlls.
Provides NuGet Gallery - almost 4.5K (at the time of writing) packages are at your disposal for all your development needs.
Supports binary versioning.
Integrates with Visual Studio.
Integrates with your build.
Integrates with your build server (only TeamCity at the moment of writing).

This tool's like a dream come true. So, what can I mention as downsides? Most of them are downsides of the NuGet Gallery, not NuGet itself. The Gallery is a young and relatively small project (just for the sake of comparison, Maven Central is 6 years old and contains more than 290K artifacts) and, as such, it has its downsides:

The content of submissions to the Gallery is (almost) unverified. Everyone can register, get the API key and start uploading whatever they like. Scary, isn't it? (yes, very scary).
Being public, NuGet Gallery can’t be used for inter-team packages exchange. Private Remote Feeds are the recommended solution. Next we'll see if it is good enough.

Working with NuGet Remote Feeds
Remote Feeds, introduced in NuGet 1.4 are crucial need for any development team. It serves a dual purpose: it allows sharing 3rd party packages that aren’t available on Gallery (or even replaces the Gallery for those who can’t trust it) and it serves as a target for internal deployments - both for team collaboration and for other usages, as making packages available to QA, or even serving them to the customers from the outside world (by using Chocolatey, for example). If that’s so right, what’s wrong? Here’s what:

You saw it coming: It isn’t a proxy. You know the score by now.
It can’t aggregate. The NuGet Remote Feed exposing one monolithic repository: the one you have on your machine. It can’t aggregate NuGet packages from remote repositories, or expose number of local repositories (separated for security reasons, for example).
You can’t attach your own metadata. Let’s say you want to annotate some package with compatibility information (e.g. works with certain browsers). No, can’t do.
The repository is very simplistic. It doesn’t provide any web interface; it browsable and searchable only from a client - be it Visual Studio or the command line interface (pretty basic by itself).
Even the VS search interface is very basic (all you have is arbitrary sorting and free text search). It should be enough for starters but lack of searching inside the packages or by properties (from the previous bullet) will bite you eventually.
The security scheme is even less than simplistic. All that's required to authenticate a deployment/delete of all the users at once is an API key. What about separation of duties? Some users should only be able to read, others only to annotate with metadata (QA team that tests compatibility in my previous example), and only small subgroup - to deploy. The all-or-nothing scheme is definitely insufficient.
Storage format is suboptimal:
- The packages are stored on the filesystem in a naive simple format. That fine for small repository,but as you grow, you'd expect storage which more optimized for binaries.
- The metadata is not indexed. Again, fine for small repo, troubles are foreseen when it comes to scaling.

So, is there a good alternative to NuGet Remote Feed to be your in-house Gallery for NuGet packages? We, the proud makers of Artifactory, believe there is:

Meet Artifactory
Artifactory is an enterprise-grade Binary Repository that centralizes all aspects of managing software binaries. That means that we tackle all the problems mentioned above. We are developing Artifactory since 2006. Being used by millions of users for storing, sharing and managing binaries, we have gathered great feedback from our users.
That's what we've learned:

Binary packages are different from sources (by being big and binary) and deserve smart storage.
Binary packages are usually archives (be it jars, zips, rpms or nupkgs). They should be browsable and searchable without the need to download them locally to developer's machine.
Big public repositories exist on the net, they need to be proxied smartly (variations, auditing, managing)
Users come in different flavors. Their permissions should match possible responsibilities (and in the case of binary packages they are different from other cases).
A binary repository holds critical information, it should be rock-solid, backed up, and DRP ready.
Your software ends up being a package. We know how to help you...
- build it in a reproducible manner, integrating with your build tools and your build server.
- stage it to ensure the best quality.
- distribute it to your customers.

You get the gist behind Artifactory by watching this 2.5 minutes YouTube video. If you’re not in the mood for movies (or ran out of popcorn), here’s quick recap (click on the image for full size):

As you see, instead of working with a number of NuGet Feeds (NuGet Gallery, Orchard Gallery, Remote Feeds from co-workers and from different teams) developers work with exactly one repository. It simplifies setup and daily work and centralizes management and maintenance.
The work is bi-directional, the users resolve their 3rd party dependencies from Artifactory and deploy their created packaged into it.
Now let’s add a build server to the picture (literally):

Yup, with numbers this time. So, here we go (click on the image for full size):

Developers find and fetch new 3rd party packages from Artifactory in Visual Studio. The packages are downloaded from Artifactory to the developer's machine. If the packages aren’t present in Artifactory it will look for them in remote galleries/feeds. On developer machines packages.config is updated with the list of used packages.
Developers commit their code and packages.config (but not the binaries) to VCS.
The Build server (as I already mentioned, TeamCity now supports NuGet) takes the changes from the VCS.
It builds the solution and packs the produced artifacts as NuGet packages.
During the build it fetches the needed packages from Artifactory. If the packages aren’t present in Artifactory it will look for them in remote galleries/feeds.
Once the packages are built they are deployed to Artifactory.

Built packages in Artifactory can be used by other teams (as their 3rd party dependencies), by QA for running tests and even by the end users (Chocolatey FTW), all this with fine-grained permissions and robust promotion procedures (moving a package between repositories with different visibility rules).
You know what? It deserves dedicated how-to blog post. I’ll link it here once published.
Assuming you've read up to this point, you've gathered that starting from Artifactory version 2.5.0 we are proud to serve the .NET world with full NuGet support. We can proxy any remote NuGet feed (starting with NuGet Gallery, of course), we can host the packages that aren’t found on any remote NuGet feed, we can host the packages you produce and we can aggregate any number of repositories of any kind under single a URL. We provide you with an awesome UI for configuring your repositories, browsing and searching for your packages. We also feature smart storage that enables attaching searchable metadata on top of your binaries. We can do it all on the cloud with our SAAS version.
Hopefully , you're convinced by now and probably looking for the download link on our site (here’s it, BTW, click on “Evalution”). If not, give it a try by playing with our live demo. Look at the nuget-gallery cache: that’s how we proxy the NuGet Gallery. You’ll find some of the packages saved locally; once you've selected a package, you’ll see all kinds of information about it: its name and size, who deployed it to Artifactory, where it came from (from NuGet Gallery, naturally for this is the NuGet Gallery cache) and the operations you can perform on this package (as anonymous the selection is naturally limited). Clicking on the triangle in the tree will open the package and let you dive into its content, including downloading specific files from the archive:

We, at JFrog, believe that Artifactory is the missing piece of the puzzle for a robust, agile .NET dependency management, which can make the development process easier compared to other alternatives.
We'll be happy to receive any insights, thoughts and comments on the ideas presented in this blog and/or your experience using Artifactory together with NuGet.

Artifactory - Community Talks

2011-11-01T15:51:00.000+02:00

We’d rather have our customers and users be the gauge of how well Artifactory met their Continuous Integration needs, rather than have you listen to our opinion.

These are busy days for the froggers getting ready to release 2.4.
The buzz is out there and proudly we can see more and more talks that cover Artifactory as part of their CI case-studies.

Following the DUKE CHOICE AWARDS in which Oracle honored us at JavaOne 2011 with the DUKE for "Innovative Tools for Developers", we would like to share with you these two great video presentations:

Building for the Cloud - Carl Quinn of Neflix talks at JavaZone and covers Netflix pipeline using Jenkins and Artifactory.

Building for the Cloud from JavaZone on Vimeo.

Agile Application Lifecycle Management - Author of Agile ALM, Michael Hüttermann, JavaOne talk. Michael describes how to execute an agile release management process using Jenkins, Artifactory and Maven.

Enjoy Artifactory!

The Frog Who Turned into a Prince

2011-10-17T14:56:00.001+02:00

JFrog’s Artifactory Wins JavaOne 2011 Duke's Choice Award

3 years from the day it was founded, JFrog was recognized by Oracle and the Java community for Innovation Java Tools for Developer with Duke’s Choice Award at JavaOne 2011.

This is not a marketing announcement nor a techie or an executive post, this blog is about how we felt the morning after we became Dukers’.

It’s been more than a year since I posted “That's what community for”, I was sure that I felt the power of the open source community, and than came DUKE...

It was Saturday around 2:00am, few weeks before JavaOne that I got the mail from Michelle Kovac of Oracle with the fantastic words “you're in the winners list right now for JFrog”. It took me 3 times to read the mail, and to actually hold the DUKE a few weeks later - to believe it!

On Sunday evening, Oct. 2nd, at the JavaOne Open House, Oracle welcomed and celebrated the 2011 Duke's Choice Award winners, JFrog Artifactory was one of them. Together with 9 other innovative Java projects, we "the froggers” stood there and felt like we got kissed and turned into a prince.

Now, you might say - OK this guy took it a bit too far...
So Imagine that you have been working like crazy for the last years (6 years since Yoav Landman, our CTO, first launched Artifactory and 3 years since we founded JFrog), your team beating any time-zone gap, and writing excellent code. You know that you are a game changer in the way people write code and you actually solve the developers pain, but still hear questions like: “So what is a Binary Repository Manager ?!” And “what do we need it for, if we use a version control ?!”

Imagine that during the last several years, you write open source based plug-ins to integrate with Artifactory’s ecosystem - just to keep Artifactory “open” and to keep the freedom of choice in the developer’s hands, and you know that you are fighting giants with that approach.

Imagine that you have been travelling all over the world “preaching” how your Continuous Integration process should look like and how you should build, deploy and distribute your software, and little by little you see followers doing as you advised.

...and then, finally you got this huge hug from the community through Oracle.

I know its better to play the cool CEO, but JFrog is all about soul and I tell you it was (and still is) awesome!!!

Fred Simon our Chief Architect tried to expressed that in the live Java Spotlight podcast - in 2 minutes talk, but we know it took much longer to come up to this point of recognition.

A week after JavaOne, we are back to ground and working hard to release Artifactory 2.4 with lots of improvements and new cool features like P2 integration to proxy and host all your Eclipse plugins via Artifactory and YUM repository for distributing RPMs directly from your Artifactory server acting as a fully-featured YUM repository.

We got the DUKE CHOICE AWARD from Oracle and we are very honored and excited, but I keep remembering the words of one of the JUG’s leaders from the other night at the conference “you’ve got a great tool and a great team - you deserve it” - that’s more than a trophy, this is what we are here for, it’s our reward and we are grateful!

We are very pleased to receive this acknowledgement of our technological achievements by Oracle and the community. JFrog strives to provide our users and customers with advanced technology solutions that address their daily CI needs and driven by it.

I once wrote “community isn't a flock” now I know it for sure!

Thank you , Enjoy your build, Enjoy Artifactory.
See you at JavaOne 2012.

About JFrog:
JFrog is the home of Artifactory – world’s first Binaries Repository Management Solution. As the first company to provide the software market with a Repository Management Solution, JFrog has established itself as a technology leader in the industry and aspires to continue setting the standard moving forward.

About Artifactory:

With Artifactory, you can gain full control over all the build artifacts and third-party dependencies your organization uses and even share repositories throughout your interdepartmental and/or multi-team environment no matter where your team members may be located.

Built on top of the Java Content Repository (JCR), Artifactory supports extremely high concurrency and offers unmatched data integrity.

Artifactory is also tool that offers an open source, Pro and SaaS (cloud based) versions.

More about the DUKE adventure at:

http://blogs.oracle.com/java/entry/and_the_winners_are_the
http://www.youtube.com/watch?v=Lrq4UBGgQpg
http://java.about.com/b/2011/10/06/dukes-choice-award-winners-2011.htm

Jenkins New Maven and Gradle Release Management and Why You Should Look Into It?

2011-06-02T21:13:00.017+03:00

Jenkins now offers a new robust solution for release staging and promotion in the form of the Jenkins Artifactory Plugin. This solution is fast, lightweight and works for for BOTH Maven and Gradle builds

The Need for a Better Release Alternative

If you are using Jenkins (or Hudson, before) then releasing and promoting build artifacts isn’t necessarily a flawless experience or may not be completely solved. To go over the main reasons of why this is so, I would like to look at Maven and Gradle builds separately.

Maven and the M2 Release Plugin

First, Maven. Jenkins already has a solution for releasing Maven artifacts in the form of the m2 release plugin. The plugin is essentially a wrapper on the client-side Maven release plugin. When it executes it runs an out-of-process Maven build with release tasks. This is natural for a plugin that was designed to run by a user on his machine, but running it server-side has a couple of serious issues:

The m2 release plugin invokes its own Maven instance that may be of a different version than the one defined for the job. Jenkins, not controlling this build, will not record any information about the release artifacts and will not archive them
The plugin also uses different SCM settings and credentials than the ones defined inside Jenkins. This double-configuration is error prone, and makes reproducibility difficult
Rollbacks are hard. With no built-in automatic rollback, when failure happens in the release flow, rollbacks become a manual task. Without a CLI, and given that the run is not interactive, this becomes even harder. Moreover, SCM tags are created with credentials that users may not have access to
It is unnecessarily SLOW. In fact, a release flow involves 3(!) full clean/compile/test cycles (you can read more about this in this blog post)

Gradle
If you are using Gradle, you are already driving your builds with elegant Groovy model+script. You may want to take advantage of that and enrich your builds with custom release logic via a Gradle plugin, which may be enough for your needs. However, there are some drawbacks to this approach:

You will need to duplicate the plugin configuration in each new project and introduce release configuration in your build logic. If you run the release from a developer machine, this is fine. However, for the most part this is not the case and releasing from the CI server with greater control and traceability calls for the configuration to be part of the job
With no strong concept of release “action” that Jenkins can understand, there can be no indicators in Jenkins or in the artifacts repository (Artifactory in this case), that a build was in fact released and that artifacts are part of a release
As with Maven, rollbacks may require manual intervention and running a release build on the server will not reuse Jenkins job’s setting like SCM details, resulting in dual configuration

Meet the Jenkins Artifactory Plugin

Given all that, we at JFrog, decided to pick up the glove and develop a new release and promotion alternative. The Jenkins Artifactory plugin provides a release management plugin that is:

Lightweight and fast - only 1 compile/test cycle is required
Has close integration with your Jenkins exiting configuration, so it reuses the SCM and build settings. The plugin directly integrates with the execution of the Jenkins Maven plugin and the Jenkins Gradle plugin
Offers very robust and reliable rollback - the plugin is primarily focused on working correctly with GIT and Subversion, respecting the differences between the tools and how tagging and branching is handled in VCS compared to a DVCS, so we don’t have to compromise on a common denominator
Provides finer control over which properties (in Gradle) or POM versions (for Maven) should change upon release and which ones should also change for the next development cycle
All commit comments are adjustable and can use Jenkins parameters

When used in conjunction with Artifactory for managing your build artifacts the plugin also -

Offers advanced control over staging & promotion flow - build artifacts can be staged to a certain repository, and staged artifacts can be either released or rolled-back directly from Jenkins UI with changing of the release status in Artifactory without requiring a rebuild
Audits all release and promotion actions
Makes release builds fully traceable by deploying a build-info json object along with the binary artifacts, with information about: environment vars; all artifacts produced; all effective dependencies used; and even all licenses used by those dependencies
Allows users to attach custom searchable properties to release artifacts upon deployment

Judging by user feedback, the Jenkins Artifactory Plugin really manages to make a difference by providing Maven users with a better alternative and Gradle users with an integrated Jenkins-side solution, for easy staging and promotion experience which is fast and reliable.

Give it a try now and see for yourself :)

The Jenkins Artifactory plugin can be found on the Jenkins Plugin Center, with the release management specific page here. All sources are on Github under the Apache 2 license.
Expect similar release management support from JFrog in the next versions of the TeamCity Artifactory plugin and the Bamboo Artifactory plugin

The Future of CI at JAX Conf

2011-05-29T15:16:00.003+03:00

The Future of CI at JAX Conf - San Jose June, 2011
by Fred Simon

These are exciting days for Continuous Integration users and for us - JFrog Artifactory team.

Following the successful CI Summit held in LinkedIn HQ last month, we are now heading to the San Jose JAX Conf!
This year, for the first time, the great European JAX conference is going to the Bay Area as JAXConf.
It will be an amazing gathering for our (JFrog) ecosystem.Many players that keep pushing the limit of Continuous Development, Build, Integration and Deployment, with their tools, knowledge and case study will be there:

Kohsuke Kawaguchi from Cloudbees and Ken Sipe from Gradleware, are getting ready for “Jenkins & Gradle Hackathon”. It’s promising a lot of great features for Gradle and Jenkins users. We will see how our great latest Gradle-Jenkins-Artifactory plugin and clean DSL will improve your day!
Of course Kohsuke Kawaguchi will present “Continuous Integration with Jenkins”, and it’s always a pleasure to hear and see Kohsuke express his view of CI and how Jenkins is moving the field forward.
And Ken Sipe will be “Rocking the Gradle” and show this amazing new build tool: Gradle.

The current growth of CI and DevOps is really impressive! The demand is so big that a full afternoon will be dedicated to CI, “The CI Circus” as a free community event.
And there Carl Quinn from Netflix, will present the amazing platform they setup “Netflix in the Cloud”, using Jenkins and Artifactory!

A lot of amazing talks about Continuous Integration, DevOps tools, and automated ALM will be presented:

“Git Going with a Distributed Version Control System!” by Matthew McCullough, Tim Berglund: The push for a VCS that can answer the need for very short cycles is very strong. Today, Git is really unlocking most of VCS bottlenecks. But it’s well known: “I know git can do it! I just don’t know how to tell it what I want!”. This full day tutorial should help with that issue.
“Improving the technical quality of your Java project” by Alexander von Zitzewitz from Hello2morrow, will present how to keep project quality over time.
“Managing Software from Development to Deployment in the Cloud” by Harpreet Singh, Vivek Pandey from Cloudbees: They will present the next generation platform to “build and deploy your apps in the cloud”.

And of course, I’m looking forward to present “You Killed My Build! Prepare To Die!”, were I will demo a state-of-the-art integration between Jenkins, Gradle and Artifactory. The setup support build isolation, release staging and promotion, and simplified developers on-boarding.

Hope to see you all there!

Thinking in Gradle!

2011-05-23T14:28:00.002+03:00

Since my first encounter with Gradle and Hans Dockter (TSSJS 2009 in Las Vegas), I slowly (but surely) started to use this new build tool in many environment and projects.
Today, I’m hooked and I don’t think there is a better way to build!
But, the main issue I encountered is how to convince other that Gradle is the good way to go? It took me time to get the hang of it, it took even more time to understand what Hans meant when he says: “What’s important is the model!”.

Here is how I describe my “Thinking in Gradle!”

After many years of Ant, and Maven brain washing, the main paradigm shift that I needed to understand the power of Gradle is: Gradle let you write code that create a dynamic model of your build.
I needed to stop thinking in terms of declaring properties (Ant), or POM static model, but an aggregation of executable blocks (Groovy closures), that will create the exact model of what are my projects, task, dependencies and products.
I don’t need to “extends”, I don’t need to “exclude”, I don’t need to “override”.
When playing with static declaration of your build model, the way to avoid repeating yourself is by declaring what’s good for 80% of your modules in a super POM, and then adding skip or repeating detailed plugin configuration for the other 20%.
In any case, you are repeating yourself a lot, and you always try to change your code or module layout to fit the less resistance of your build tool.

Your “static” build model is freezing your ability to re-organize your modules as they should be.

With Gradle, the build model is created from executable Groovy code. So, nothing, I really mean nothing is static.
It’s extremely disturbing for newcomers. I want my properties, my XML, my declarations :)
No! It’s only code, dynamic Groovy code!
The model will emerge from Groovy collection closures (apply this to any model element that matches), some “if”s when needed, and a lot of beautiful GStrings for expressing dynamic values. You cannot keep your thinking of static XML, when you write your Gradle build!

OK, you may think that the dynamic part is just your mental representation of a POM hierarchy and dependencies.
Well, in Gradle the execution task graph (every little plugin execution of your Maven lifecycle, which is extremely static and a nightmare to modify) is also dynamic.
It means that the way you chain the tasks that will be executed are also defined in code, not in XML :)

And of course, the part that everyone expects: The execution block of a task is also in Groovy (or Java) code.

Conclusion
Until I let go of my old concepts of static task dependencies (Ant), and static project model (Maven), I missed most of the beauty and power of Gradle!
Hope this will help others.

The First Continuous Integration Summit - Video Available!

2011-04-20T10:41:00.000+03:00

On April 7^th, we held the First Continuous Integration Summit at LinkedIn HQ in the Silicon Valley.

We were overbooked a week from opening the event registration page! This enthusiastic response, in addition to the feedback we received (and still getting) from the community, illustrates the great appetite for gaining and learning more about CI technologies and methodologies.

Together with our partners from Gradleware and CloudBees, we have managed to stir up and arrange this event for all of you who use and explore Continuous Integration tools and technologies.

We couldn’t be more excited to be part of the very first Continuous Integration Summit, an opportunity made possible largely thanks to LinkedIn, who hosted the event and now provides us with this video recording of the sessions:

LinkedIn Case Study – present and future of LinkedIn's codebase build
"Who Moved My Module?!" – Yoav Landman, JFrog CTO and Artifactory Creator
"Enter the Gradle" – Hans Dockter, Gradleware CEO and founder
"Status of the Jenkins Project" – Kohsuke Kawaguchi, CloudBees architect and Jenkins Project lead

We already started planning the next CI Summit – stay tuned ….

Enjoy ;)

JFrog Team

Artifactory Vs. Nexus The Integration Matrix

2011-02-24T22:38:00.000+02:00

Are you stuck with your vendor stack ?!

The evolution of Continuous Integration, build tools and build servers in the past years has been very impressive. The amount of projects (open source, or not) and tools that were launched and later adopted by the community (Maven, Gradle, Jenkins/Hudson, TeamCity, Bamboo, and more...) shows that this is the future of software development and this is how people will build and package their applications.

One of the greatest advantages, especially in the Java world, is that development teams have the freedom to choose and build modular environments by integrating the tools that they like, need and that were adopted by their organization.

This is why our users build their projects using Maven, TeamCity, Ivy, Hudson, Gradle, Bamboo and recently Jenkins ;). Bottom line is, you should not be concerned about the ability of the tools to integrate while designing your CI stack!

BUT, things are changing. Now even more with the fork of Hudson CI (currently owned by Oracle), Artifactory has come to be the only Binary Repository Manager that gives you the real freedom to choose!

We are not presenting ourselves as a community “rescuer” under the open source hood, or try to take ownership over successful projects that were developed by the community (such as Maven or Hudson); we are developing Artifactory in a way that gives our users the ability to work with their tools and technologies of choice within the build ecosystem.

During the past year, JFrog’s team has been developing for the Artifactory users a number of open source plug-ins that are tightly integrated with the worlds leading tools and vendors.

The following comparison tables are more than just a list of features - it is what we envision when we think about the software development tooling, and how we ensure that our users keep their freedom of choice:

Build tools integration

Build servers integration

Some features are part of the pro versions of Artifactory and Nexus.

Now, it’s easy to say “We will provide you with one full stack”, but isn’t that like saying “We know better what’s good for you”? Is it preferable over giving you the freedom to choose?!

To Build or Not to Be - Seminar Videos

2010-07-26T10:06:00.008+03:00

JFrog's Continuous Integration and Build Seminar "To Build or Not to Be", took place on July 1st, 2010 and was a big success.

The sessions of Kohsuke Kawaguchi creator of Hudson and CEO of InfraDNA, and Hans Dockter creator of Gradle and CEO of Gradle Inc are now available online.

Watch now the videos of "Gradle - A Better Way To Build".

and "Doing More with Hudson".

Enjoy!

The case study of JBoss Repository Manager

2010-05-11T15:54:00.003+03:00

Most of the issues encountered by JBoss developers with their new build infrastructure are discussed openly here. This is an important source of information about the problems encountered with a Hudson, Maven and Sonatype-Nexus integration.

Since we (JFrog) worked on Hudson, Maven, and Repository Manager environment for many years, we provided some feedbacks to JBoss.

The main thing is: We are not promoting or supporting any particular build technology and so when Maven does not fit we are not afraid to say it as it is.

This case study exposes vividly the differences between Nexus and Artifactory!

Here are the issues I'm talking about:

Mod dav or http(s) deployment URL
Managing the distributionManagement XML tag in pom.xml
Using password for Maven deployment
Timeout and performance issues
Maven metadata XML issue
Maven deploy failures
Creating and verifying checksums

1) Mod dav or http(s) deployment URL

To solve the timeout on deploy issue for richfaces, the recommendation was to use pure https instead of dav:https as url deployment.

Here are all the caveeat you need to know about Maven deploy wagons:

There are 3 main kind of Wagon used with Maven: Lightweight http (the default today), Heavyweight http (the default for older Maven versions), Dav.
Dav uses transactions when deploying, it has a better management of memory on the client side, and has a bug calculating checksums. Dav is a must when deploying big files. Now, the issue with dav is you can get timeout due to transactions, so the recommendation is correct!
Lightweight http(s) included in Maven, cache the whole byte array in memory so cannot work for big deployment.
Dav and Heavyweight protocols suffer from a default configuration that transfer each deployed file twice over the net. This is due to standard HTTP protocol always trying to send a request without authentication, then with it. Check this to find the way to configure preeemptive http authentication in Maven simple http client.
Maven is caching credentials information based on URL. It means that if you are authenticating the read queries, the same read credentials are used for deploy! See "Configuring Authentication" here.

Basically, looking at the above options nothing is satisfactory with standard Maven.

To solve this issue, we (JFrog) created a build client plugin that manages deployment of the artifacts using Apache commons http client with preemptive authentication and optimized response time. For info: This client is used to deployed full DVD images of 4.5Gb to Artifactory multiple times per day, from multiple Hudson servers and agents...

2) Managing the distributionManagement XML tag in pom.xml

To manage correctly all the above parameters, and the deployed URL of the exact repository location for each projects, you need to configure the distributionManagement tag in the pom.xml. This generates a lot of friction, and makes rebuilds of SVN tags almost impossible.

For good reason, JBoss wants to split the repository manager into multiple servers and local repositories, so each projects will deploy their artifacts to a dedicated repository.

Since this distribution is a progressive change, all the previous builds and SVN tags are obsolete and cannot be reproduced...

We believe that managing deployment configuration parameters should be done outside Maven, and that's why we have an Artifactory-Hudson plugin that does exactly this. Your pom.xml are clean of repository manager URLs and you can tuned deployment parameters (Timeout, http or https, ...) in Hudson outside Maven.

BTW: The Artifactory-Hudson plugin provides a lot more features like full bidirectional traceability between Artifactory and Hudson for all the artifacts produced and used during each hudson job execution.

3) Using password for Maven deployment

Like described here having encrypted password in your settings.xml is a little annoying but it works. Now, you need to know that it still forces you to use https since the clear text password will pass on the net. Now, the issue with Repository Manager password is that all kind of automation scripts beside maven needs them (Hudson, buildr, Ant, auto-backup, rsync, ...), and so you always find yourself in a tricky situation, see this and that.

To solve this issue, we implemented an encrypted password system that works as follow:

Artifactory generates an asymmetric key pair for each user,
Entering your clear text password in the UI gives you a block of the settings.xml with your personal encrypted password,
The above encrypted password can be used anywhere (not only maven), and allows only REST API access (maven deploy and so on) not UI access,
With the encrypted password, https to do maven deploy is not a must anymore => a lot less load on the servers,
Stealing the encrypted password or the private key (kept in Artifactory DB) has minimal security impact compare to full clear text decryption of all the passwords used on the JBoss Hudson server,
You can enforce to always use encrypted password for REST and maven deploy.

4) Timeout and performance issues

This issue is related to the dav mode, the JVM memory parameters, OS open file handles tuning, but also to the next issue about maven-metadata.xml files.

From our experience, one issue with latest 1.6 Sun JVM is the miscalculation of young memory size. So, when updating/upgrading your server you should verify that -XX:NewSize and -XX:MaxNewSize are at least set to 2/3 of the total heap size. Artifactory (like Nexus) needs a lot of memory space per request (managing files), and so the young eden space of the JVM is the most important parameter. Just be careful to set a good multiple of mega bytes has the JVM has a bug moving memory blocks from young to old gen space.

From what I read here, this is what I recommend: "-server -Xms1g -Xmx1g -XX:PermSize=128m -XX:MaxPermSize=128m -XX:NewSize=512m -XX:MaxNewSize=512m"

5) Maven metadata XML issue

This issue is a little tricky to explain, I hope I'll be clear :)

For each maven pom or jar file of a SNAPSHOT version, maven needs to 2 maven-metadata.xml files. I'll take the richfaces issue as an example:

Deploying: org/richfaces/ui/modal-panel/3.3.4-SNAPSHOT/modal-panel-3.3.4-SNAPSHOT.pom needs to update:

org/richfaces/ui/modal-panel/3.3.4-SNAPSHOT/maven-metadata.xml to update the timestamp and or the latest unique version number of this version,
org/richfaces/ui/modal-panel/maven-metadata.xml to update the list of available SNAPSHOT versions and the latest one created.

The issue is that the maven client is doing this complicated work, combined with the fact that maven-metadata.xml files are very repeatedly requested by maven to build its version resolution scheme.

So, in this example, for every module of richfaces 3.3.X-SNAPSHOT, maven deploy the jar (twice due to issue 1), the pom, the sources, read the previous values of maven-metadata.xml on both folders, then write them down (twice due to issue 1). Of course due to dav and the heavy read on these files, you get timeouts.

The performance tuning planned here will help solve this issue since the reads will be offloaded to nginx.

Be aware, you will hit a bigger problem due to the above design: Repeated corruption of maven-metadata.xml files!

Like everyone, most of JBoss projects have upstream and downstream dependencies in Hudson. For example: richfaces-jsf2 depends on richfaces. So, it is common that multiple separate SNAPSHOT hudson job (let say richfaces-3.3.4-SNAPSHOT and richfaces-3.3.5-SNAPSHOT) will get executed in parallel. As you can see from the above maven design, the calculation of maven-metadata.xml files will almost certainly fail.

To solve this issue, our deployment system (in the hudson plugin) knows that Artifactory is a smart repository manager and so sends only the pom, jars and verified checksum to the repository manager. Artifactory is then queuing maven metadata recalculation tasks asynchronously, and so avoiding parallelism conflicts and timeout. Implementing this mechanism was far from evident, but we have today integration tests deploying 5 different snapshot versions concurrently (and repeatedly) without a hitch :)

6) Maven deploy failures

As seen here, on failed Hudson jobs trying to deploy to Nexus, you end up with half deployed projects. It means that until the successful build 572 of richfaces, the repository contained a random list of good and bad jars and pom for the same project/version. It is recommended to actually do the deployment only until the full build is successful.

Deployment needs to be a post build action of Hudson (like it is done in our plugin).

7) Creating and verifying checksums

Since maven is calculating and deploying the checksum to JBoss Nexus instance, you have general issues like:

Playing with dav may have generated a lot of bad checksums that you don't know of;
There is no bad checksum verification on deploy, and so the build will be successful but users will get a wrong checksum error;
Since the correspondance between hudson and maven checksums could not be trusted you'll lost traceability between Hudson and Nexus.

For info: Artifactory is capable of failing the deployment if checksums does not match, and has many checksums policy configuration.

I hope Red Hat will solve their build infrastructure issues for the good of JBoss developers and users :)

Building an Enterprise Repository with Artifactory

2010-03-03T17:17:00.046+02:00

*The content of this blog is a translation of a blog posted in Portuguese by Diego Pacheco.*

I clearly recall experiencing DLL hell while working predominantly with Microsoft products. We suffered from dependency issues back then and we still suffer from them today.

When I started working with Java I suffered from a similar development concern. The specifics are a little different, but the essence of the problem is the same – dependencies. Java developers know this experience as JAR hell.

It's amazing to me that companies still struggle with these issues in 2010. There are good solutions for solving dependency problems. Two such solutions are Apache Maven 2 and Apache Ivy. Both these Apache solutions provide dependency management.

It's common to see a folder called *lib* in the classpath of Java applications containing more than 100 jars. Often, 70% or more of these jars are completely useless, never used, never have been touched by the system ClassLoader, or will seldom ever be accessed by any system or user-defined ClassLoader. Still, they remain in the build. But why? Why leave so many many jars to bloat the application? For two reasons:

Dependency Problems: Everyone has probably seen the famous ClassNotFoundException at one time or another. Often, this problem is solved by copying all the jars from the application server and framework distributions we use to the good old *lib* folder. Of course, this is not an elegant solution, and it is not a good solution, but is widely used because it is easy to implement and it saves time. The problem you get to live with is that you're hoarding a bunch of jars you don't need and the size of your application will swell.

Note: The more robust application servers like Websphere for instance, let you create shared libraries. You don't have to pack all the jars with your distribution (i.e. ear, war, jar) because you can let the server put them in the classpath.
Ignorance: Many companies still don't use any dependency management solutions. As a consultant I've seen this a lot, in Brazil and abroad. Maybe some companies believe their developers are doing great without some fancy management solution. The rest of us will be investing in dependency management solutions. Personally, I like either Maven or Ivy for dependency resolution.

Using a Dependency Manager

If you still don't make use of a framework for dependency resolution, I recommend you to start using one ASAP.
Either Apache Maven 2 or Apache Ivy will provide you with benefits like:

Transitive dependencies: making it easier to find those elusive dependencies
Versioning: Using the "dump it in the *lib* folder" approach there is no control over which version is needed or which version is in production or development. A dependency management framework provides you with fine granularity control without going mad.
Automatic downloading: With a dependency management solution, you don't have to search for dependencies, download them, and put them into the classpath of your application. This is a huge advantage.
Versions update: Here's another big headache handled for you, especially if you have several applications that make use of a jar that you develop internally. Good dependency management keeps you from going over every application to perform version changes by hand. This will save you some real hard work and a huge amount of time.

I hope that you are already convinced that having a dependency management solution like Ivy or Maven is a good idea. Maybe you use one or the other already.

Often, companies that use Maven or Ivy forget to set up a good corporate repository solution behind it all.

Building a corporate repository with Artifactory

I've used Java corporate repository solutions such as Nexus and Archiva in the past. Recently I use only Artifactory and I've never had a reason to look back. Artifactory is a very elegant solution to dependency management. The main issue I had with Archiva is that it downloads and resolves dependencies rather slowly.

Firstly, I recommend Artifactory because it does a much better and faster job of downloading and resolving dependencies.

Secondly, I heartily recommend using Artifactory because it provides a comfortable web UI for the management of your third party jars (solutions and plugins) like Spring, Hibernate, JBoss, as well as managing those that are produced in-home by your company. This separation can be done using Artifactory's repositories.

Thirdly, I strongly recommend Artifactory because it provides separation and control of the many versions of development and production material. Most of the time people use Artifactory only as a proxy solution that helps you save some traffic from your company's bandwith, but it's actually much more.

Dependency Control

Artifactory allows you to control the permissions of users and groups. You decide which applications can consume or use certain repositories and jars. For example, you can restrict the use of Spring only for some projects. I wouldn't do that :) but it can be done. Artifactory also provides you with a much better way to manage your internal dependencies and the solutions you write by letting you limit and centralize access to those dependencies. You may never miss a dependency again.

M2 and Ivy Consolidated Repository

The figure on top shows that by using *virtual* Artifactory repositories you can serve both Ant-Ivy and Maven clients. The coolest thing is that you can serve both clients from the same repository.

How is this possible? Because Artifactory is flexible.

Artifactory works under the Maven 2 dependencies standard. This standard includes:

GroupId: the group of the solution, macro grouping of a module, or a providing company (e.g. org.springframework)
ArtifactId: represents or identifies the available / consumed artifact – could be the name of the jar like in our old *lib* folder (e.g. spring)
Version: the version of the jar (In Maven when we see something like 1.0, normally it indicates that the solution is a production release and if available, you should use GA. When a solution is under development, a SNAPSHOT version is used; sometimes followed by the date and time of the last build.)

With Ivy things work pretty much the same in the attribute organisation maintaining a respective equivalence to Maven 2:

Organisation: similar to Maven GroupId
Module: similar to Maven ArtifactId
Revision: similar to Maven Version

The big difference between the two Apache Dependency solutions is that Ivy uses a regex-style pattern for dependency resolution, and Maven uses a predefined pattern for dependency resolution. It's a fact that Ivy gives you more flexibility, but also demands more work. What I suggest is to use the same Maven pattern for Ivy, so deployments to Artifactory will be consistent. This way you'll be able to use the same repository for Ivy and Maven.

Configuring Ivy to download and deploy to Artifactory

To accomplish this, all you have to do is use this XML file I've named ivy-artifactory-settings.xml:

1<ivysettings>
2    <settings defaultResolver="public"/>
3    <credentials realm="Artifactory Realm"
4                 host="seu_host_do_artifactory"
5                 username="admin_user" passwd="admin_password"/>
6    <resolvers>
7        <ibiblio name="public" m2compatible="true"
8                 root="http://seu_server:8080/artifactory/
9                 seu_proxy_repository"/>
10        <url name="publish_artifactory" m2compatible="true">
11            <artifactpattern="http://seu_server:8080/artifactory/
12            seu_repositorio_release_repository/[organisation]/
13            [module]/[revision]/[artifact]-[revision].[ext]"/>
14        </url>
15    </resolvers>
16</ivysettings>

Do not forget to use the pattern [organisation]/[module]/[revision]/[type]s/[artifact]-[revision].[ext] to resolve the dependencies. By the time you are publishing your jar via Ivy, you can use a task to make the Maven POM based on Ivy's dependencies. It should be something like this:

1<ivy-makepom ivyfile="${ivy.xml.file}"
2             pomfile="${basedir}/dist/${ivy.organisation}/
3             ${ivy.module}/${ivy.revision}/
4             ${ivy.module}-${ivy.revision}.pom">
5    <mapping conf="default" scope="compile"/>
6    <mapping conf="runtime" scope="runtime"/>
7</ivy-makepom>

Taking into account that you created your jar in the dist folder and that it follows the Maven 2 pattern, you will notice that it's located within the hierarchy of organisation, module and version folders and that the jar is tagged with the module name and the version.

Consuming and publishing your artifacts using Maven shouldn't put you to any trouble, since Ivy now follows the same pattern as Maven.

So no matter if the project uses Maven 2 or Ant with Ivy, by using Artifactory and following the patterns and tips from this post, you will be able to use the same repository for everything.

Best regards until next time.

So you've decided to configure a remote repo and avoid headaches?!

2010-01-14T14:26:00.005+02:00

Background
There are a lot of public Maven 2 repositories out there (repo1, JBoss, SpringSource, etc.). When setting up your a repository manager for your organization, configuring remote repositories can be one of the most difficult tasks. Finding the correct URL for those remote repositories, and more importantly, defining the correct include/exclude patterns for artifacts, is not always a trivial thing to do.

However, we at JFrog believe that if you are using Artifactory as your repository manager, this task can become a snap using remote repository configuration sharing!

Remote Repository Provisioning

To share the configuration of your remote repositories all you need to do is configure a remote repository once. Then all other Artifactories that you allow access can connect to yours and pull the remote repository configuration via REST and they are all set and ready to go in a matter of minutes!

Artifactory allows you to define include/exclude patterns on the repository level (as opposed to group level), which is important for repository configuration sharing, since it deems each remote repository responsible for a predefined set of artifacts.

The Process

The process is split into two separate parts: sharing and importing: First, you must choose which repository configurations you want to expose (you don't want to expose repositories with sensitive information) and then other Artifactories simply pull the configuration.

1. Sharing: choose which repositories to expose

Go to your Repositories page and select which remote "repo" you would like to share. Then, in the "Advanced" area of the panel you need to select the "Share Configuration" check box and click Save. It's that easy.

2. Importing: adding repositories to a new artifactory instance

Now let's go to the other side. Let's say that you have a brand new instance of Artifactory running. In the repositories admin page you can see the default list of remote repositories. You can select these predefined common remote repositories (JBoss, SpringSource, Java.net, Google, and so on).

If you wish to update one of the repository definitions or add a new one that is not currently there all you need to do is click the "Import" button. Enter a remote Artifactory URL, or just use the default one that points to "http://repo.jfrog.org", to get a list of most common well-known repositories.

Now, click "Load".

You will get a list of readily available repositories is at your disposal. Simply check the ones you want, modify the repository key if needed, and import. The process is quite easy and lightweight!

Assuming you are in a large company that has several Artifactories running in multiple locations, you do not need an expert to configure those repositories again. All you need to do is connect to one central Artifactory that holds all remote repositories already configured and working, and simply pull the configuration over to your side. You are ready to go in a matter of minutes and without the headache of trying to make sure that configuration matches the one on the central Artifactory.

Remote repositories may change (often leading to a domino effect in artifacts resolution). Artifactory deals with that too by allowing a repository configuration to be overridden with a newly retrieved one. You can of course rename an imported configuration in case it conflicts with an existing configured repository.

Conclusion:

Artifactory OSS version offers a powerful way to make remote repository configuration easy, eliminating redundant maintenance pains. By following a simple and easy-to-use process the entire aspect of repository sharing becomes a no brainer.

That's it—enjoy and happy building!

Empower Hudson with Artifactory - Track and Replay Your Build Artifacts

2009-12-30T22:03:00.034+02:00

Overview
In this blog, I will demonstrate how to integrate Hudson with JFrog's Artifactory repository manager to have full build-to-artifacts traceability. We will use Artifactory plug-in to deploy the Hudson build artifacts and track them back to their original build.

Keeping the history and reproducibility of code is a must-have for any modern project.
Using one of the different flavors of version control applications, you can easily reproduce the state of any point in the past using the different methods of SCM tagging.

But what happens when you want to reproduce binary products from a certain phase?
Are dependencies considered? Does anyone really remember what version of dependency X was used in version 1.0 or in version 3.1 of your application? What if you used version ranges or dynamic properties? Was the application compiled using JDK 5 or 6?

All this information can be recorded during the publication of your binaries, which is usually done by a CI server of your choice.
Your CI server has all the knowledge required in order to reproduce a build:

Information on the builds themselves
The published items
Version information
Dependencies
Build environment details

But how can you capture all this data? This is where Artifactory kicks in!

Artifactory (v2.1.3+ OSS) is open for communication from any build process to receive information needed for tracing/reproducing a build - the sender of this information is typically your build server!
The information is transferred via REST in the form of a BuildInfo JSON object and contains details about the modules, artifacts, dependencies, environments, properties, and more.
All builds and binaries are provided with bi-directional links that enable you to reproduce and analyze the impact of any action.

Presently, JFrog provides a first integration with Hudson and Maven 2. Other technology stacks are coming, but for the purpose of this blog I will use a setup of Hudson with a Maven 2 build.

So let's get "crackin'"!

We assume that your instance of Hudson is already configured to request all it's dependencies from Artifactory. This of course ensures that all your build's dependencies are cached in Artifactory and can be used for build reproducibility.

Installing Hudson's Artifactory Plug-in
To install the Artifactory plug-in, simply browse from the main menu to "Manage Hudson" -> "Manage Plugins" -> "Available" Tab, and check to enable "Artifactory Plug-in". Once the plug-in has been downloaded and installed, restart Hudson for the changes to take effect.

Now we'll configure the plug-in on a system-wide level and point it to the Artifactory to which we would like to publish the information (please note that Artifactory should be running and available at this point).
To do this, enter the "Configure System" menu via "Manage Hudson" -> "Configure System", and then configure the URL (up to the application context name, like "http://localhost:8081/artifactory"), and optional credentials (if anonymous access is enabled, you don't need to provide them). The need for credentials of an authenticated user comes from the fact that Hudson requests a list of deployable repositories from Artifactory, so you can choose the destination of your binaries at a later stage.
Notice that you can add multiple Artifactory configurations to suit your needs.

Next, we'll configure the plug-in at the "Job" level.
Enter the Job configuration by selecting your Job and clicking the "Configuration" link. Scroll down to the Post-build Actions option group, and select "Deploy artifacts to Artifactory". Once selected, the menu will expand and will let you choose the "Artifactory server" and "Target repository" to which to deploy. As implied by the field names and the "Deployer username" and "Deployer password" credentials you provide, you must have "Deploy" permission on the target repository you select.
Being able to select your deployment target from a ready-made list, which is received directly from Artifactory, helps you avoid the pitfall of configuring your "Distribution Management" with typos.

The Artifactory plug-in deploys via REST API, which optimizes the process of unique/non-unique snapshots and doesn't require credential and distribution management configuration in your settings.xml and POM files.
Unlike Maven, which deploys each module as its build is completed (which may result in a partial deployment of your project's artifacts if your build fails at some point), the Artifactory plug-in deploys only when the entire build completes successfully (much like the built-in Hudson deployer). Each deployed artifact is tagged with buildName and buildNumber properties, and finally the Build Info is published.

At this point, you can run your Job, and then view the "Console output" to see the deployment and build info publication log messages.

Artifactory's Build Management
Now that the Job is complete, the artifacts are deployed, and the build info is published, we can view the build info in Artifactory by clicking the "Artifacts" tab under the new "Browse:Builds" sub-menu.

Here, we can see a list of all the published build names and the time each was last built.

Drilling down through the build number list, we can view the general info, the published modules, and the XML representation of the selected build.
Notice that the top of the build browser displays navigable breadcrumbs that are also synchronized with a RESTful URL that provides easy access to every part of every build.
The general info tab displays the main details about the build (name, number, type, etcetera) properties that were attached, and even the option to save the published module's artifacts and dependencies as saved search results (requires the "Smart Searches" add-on).

Clicking on the name of a module displays a list of the artifacts and dependencies that are part of the selected module.

When deleting an artifact that's associated with a build, either as a product or a dependency, Artifactory will notify you of the association prior to the removal.

Promotion of published modules is also made possible by the "Save search results" actions that are available through the General Build Info tab (requires the "Smart Searches" add-on).
Moreover, buildName and buildNumber properties, allows us to manually search build artifacts through the Property Searcher (requires the "Properties" add-on).

Conclusion
Using Hudson (and others to be supported soon) and Artifactory we've:

Supplied Hudson with all the needed dependencies from Artifactory—helping us keep the exact dependencies that were used in each build
Deployed all produced binaries to Artifactory—helping us keep and promote all the products of the build
Published build information to Artifactory—helping us manage and keep track of every build, environment, product, and dependency

With the assistance of these tools and methods, you will be able to reproduce and execute a build from any point of recorded time or compare information between different builds.
You may want to visit our build integration wiki page for a more in-depth explanation of the process.

Have fun, and be careful not to break the build. ;)

The one that talks, the one that does!

2009-12-24T11:03:00.003+02:00

In a blog "Why Putting Repositories in your POMs is a Bad Idea", Sonatype "asked" the open source community to manage their Maven2 POM file correctly.

This is a good and important request, since Maven will not work correctly:

Over time (due to URL changes)
In a closed environment (no direct access to the Internet from a developer machine)
Because it will shortcut the repository manager of your choice (Nexus, Archiva, or Artifactory) for resolving dependencies—this is most important.

We are facing this problem for almost every customer that uses Maven, and most of them are using the lazy and dirty solution of "mirrorOf". It is argued in the blog that mirroring all Maven requests to a single URL is a good idea: We know it is a bad idea as it completely takes away control of isolating the source for releases, snapshots, and plug-ins!

There is however, a dramatic sentence in the blog about POM files coming from Open Source projects (it actually applies to everyone):
"The entries you have defined will be burned forever into your released POMs."
It sounds like Maven is broken by design and forever, because of all the bad POM files that already exist out there.

Since we support our customers, and they are suffering from actions that are not under their control, we decided to fix it.

In the latest version of Artifactory (2.1.3), there is now a new feature: Automatic cleanup of remote repositories declared in POMs.
You can now configure any virtual repository to automatically clean up rogue remote repositories declared in POM files.

By default, Artifactory will do it for repositories and plug-in repositories directly declared under the project POM entity, or declared inside an active-by-default profile. You can enforce a deeper cleanup that removes all repository and plug-in repository declarations in all profiles.

Using Artifactory as your repository manager means that you will never get "burnt forever" by innocent mistakes done in POM files of nice, popular Open Source projects.

"The one that talks, the one that does!"

Search-based Promotion - Staging and Promotion Finally Made Simple!

2009-11-05T13:31:00.020+02:00

Overiew

One of the greatest features of Artifactory 2.1 is the support for artifacts staging and promotion.
The idea behind this feature is that in many environments, before exposing a new release for public consumption, the release needs to go through a well-known life-cycle - the release is first made available in a staging environment where it is validated and undergoes final QA, only then it is moved or 'promoted' to a place where it can be found and used by clients.

In the world of releasing artifacts this means that when to-be-released artifacts are built and deployed into Artifactory, you want them to first to be accessible only by a selected group (staging). It is only when the artifacts have been 'blessed' and found to be release-ready, that you can change their access-level and make them available for download by a wider audience (promotion).

There are two problems with managing staging and promotion. The first problem is around identification - how can we manage and identify a group of artifacts deployed to Artifactory by the same build, in order to promote them later on as a single unit.
The second problem is about promotion composition - once to-be-promoted artifacts can be identified, what is the best way to collect them and change their visibility. Also, what if you wish to be picky about promotion. For example - what if you want to sieve test artifacts from the release or remove by hand an internal artifact that was originally part of the deployed files?

The solution described here takes advantage of one of Artifactory's coolest features - the ability to attach properties on artifacts and the ability to compose and operate on search results. It uses some features found in the Artifactory Add-ons Power Pack, but with some additional setup it can also be adapted to the open source version of Artifactory.

Identifying Artifacts Deployed as a Group

The Quarantine Way
One way of identifying same-build artifacts is by creating isolation: you create a "quarantine repository" per-build and deploy your artifacts to it. This guarantees that artifacts are isolated in a confined place where they can be identified. The next step is to make the quarantine repository visible to your staging clients.
The main problem with this approach is that it somewhat a heavyweight solution:

You need to create a pseudo repository on-the-fly, usually using a cryptic handle (such as a time stamp and the originating IP address) which makes it complicated to remember and identify later on.
For each such temporary repository you need to configure how it will be exposed under a more user-friendly repository and who will have permissions to view it.
You need to manage the repository and decide when to close it and how long it needs to be kept.

So, we decided we don't want to go this way of creating a quarantine area and introducing unnecessary maintenance efforts. Moreover, the whole concept of creating an on-the-fly repository for the sake of supporting promotion, just felt unnatural to us! What we really wanted to do instead is to be able to deploy our artifacts to a normal managed repository accessible by the QA users. Then, we wanted to find in this repository all the artifacts that were deployed by the same build and promote them as one unit to a different repository. If we could also have a way to do some manual artifacts filtering before promotion this would also be nice.

Meet Artifacts Tagging
Artifactory supports attaching properties to artifacts (and folders). You can search for property values and retrieve artifacts that were tagged with properties. So, if we could somehow attach properties to our build artifacts at build-time in a simple manner, then we can use these properties to locate and identify the deployed artifacts originated from the same build for the sake of promoting them later on.
Properties can be attached to artifacts in many ways: via the UI (the Artifactory Properties add-on allows you to attach custom strongly-types properties), via REST API using PUT requests and as a piggy-back to artifacts PUT requests using matrix parameters.
We are going to use the last method to transparently tag our artifacts as they are deployed to Artifactory by the build process. To accomplish this, all we need to do is add matrix parameters to the deployment URL, as part of our POM's 'distributionManagement' section:

<distributionManagement>
<repository>
   <id>qa-releases</id>
   <url>
   http://myserver:8081/artifactory/qa-releases; \
   buildNumber=${buildNumber}; \
   revision=${revision}
   </url>
</repository>
</distributionManagement>

Matrix parameters are a set of key-value pairs separated by semicolon (;) and are a standard HTTP way for specifying parameters (in addition to query parameters and path parameters).

In our example, we added two parameters to the deployment URL: buildNumer and revision, both will be replaced by Maven in deployment time with dynamic values from our project properties (e.g. by using the Maven build-number plugin). When building a multi-module project all the artifacts deployed from the parent project will end up having the same buildNumber and revision values. When Artifactory sees the values as part of the URL it automatically attaches new properties with the supplied key/values to the deployed artifacts.
This mechanism works with any Artifactory 2.1+, without having to install any add-on. It also works with any REST client, which means you do not have to customize Maven or use a Maven plugin - it just works out-of-the-box.

The artifacts in this example are deployed to the 'qa-releases' local repository, which is our common place for staging artifacts before they undergo final testing and approval by the QA team.

Composing And Promoting Artifacts

The Goal - Search and Move
So, we managed to have all our build artifacts tagged with properties and deployed into the staging area. Next the QA team has gone through all known issues and approved our artifacts for further distribution to clients. We now need to find a way to copy or move our artifacts to another area where they can be downloaded by these clients. In fact, we can think of 'promotion' as a fancy term for moving (or copying) artifacts to a different place where different access rules apply.

So, what we really need to do is search for our recently deployed artifact by their build number property and then move the result to another repository.
One way to do this is via REST API - search for artifacts by metadata (properties are stored internally as XML metadata) and send DAV MOVE requests to move the results one by one. This will require writing a small external script to send the relevant HTTP queries and can be done using the basic open source Artifactory version. However, there is a much simpler and more powerful way to do this inside Artifactory itself, which involves the 'Smart Searches' and 'Properties' add-ons.

Finding To-Be-Promoted Artifacts - Meet Smart Searches and Properties
The idea behind smart searches is as follows:
You collate artifacts by searching - start with performing a search, then save the search results found into a named 'search result'. A 'search result' is merely a collection of found items. You can now perform additional searches, and for each search you can choose whether to create new search results or, more interestingly, add or subtract the items found to/from any previously saved search result. You can combine artifacts from any search type (simple, GAVC, class, XML etc.) - the combinations are really unlimited!

In our case we will simply run a property search for all artifacts deployed with a certain build number (tagged in deployment with the buildNumber property). For this, we will use the Properties search user interface:

We will then save our results as a new search result called 'to-be-promoted'. We can perform other searches to add or remove artifacts from the saved results.

We can even manually tweak our saved results by discarding specific artifacts - say one artifact in the build needs to be closed source, we can discard the sources form the results (if we wanted to discard all sources we could do this easily by subtracting the results of a GAVC search for the 'sources' classifiers - see the screenshot below).

Once we are happy with our saved search results, we can go on and promote them!

Promotion - Just Move It!
Promotion is super easy - we just move the saved search results artifacts to a target, more open, repository, thus making them available for public consumption. We can choose whether to move the original artifacts or create a copy of them in the destination repository.

We can even perform a 'dry run' to see that everything moves fine and we have no errors or warnings (for example, the destination repository might not accept some artifacts due to it snapshots/releases policy, security policies or include/exclude path patterns).

Finally, we can also export the search result artifacts to the file system if we wish to use them elsewhere outside Artifactory.
When search results are copied ro moved they maintain their original metadata and properties, so any valuable information you might have on your artifacts is kept.

Wrapping Up
Artifactory is taking a unique approach towards staging and promotion - instead of artificially requiring artifact to be deployed into a quarrantine area, we simply tag artifacts with metadata upon deployment and have them deployed to a central staging repository that is accessible internally to the development and the QA team members. This process is cheap, doesn't require complicated setup and is natural to both developers and testers. If we wish to avoid double-deployments to the staging repository we can easily do it using permissions (using the delete privilege) - there no need to manually close the whole repository for this.
Once we are happy with the staging build artifacts, we can collect them by searching and simply move or copy them to another target repository where they are publicly available. In this example we tagged our artifact with a buildNumber property which we searched for later, but we could attach any properties to our artifacts and perform many kind of searches to control our promotion source (by combination of property values, by GAVC etc.) - with metadata on artifacts we are not limited by the physical repository our artifacts were deployed into!

Finally, we created a screencast showing the promotion methodology described here, in action.

Enjoy!

Maven and JavaFX, the story of TwitterFX POM

2009-05-11T23:06:00.006+03:00

JavaOne demo

Our JavaOne 2009 technical presentation TS-4388 is a demo integrating Maven, JavaFX, Artifactory and WebStart in one smooth process for the developer and site operation manager.
So, we were looking for a nice JavaFX demo with source code using external Java dependencies that will demonstrate the ease of use of Maven when using Java libraries. From the JavaFX Application contest and from some Twitter news we tried TwitterFX.

We were immediately impressed by the UI quality and slickness of this application: Bravo!
So we checked out the source code, and we saw the need for Maven on this project. All the dependent jars were included in Subversion (3.1M of jars out of a 4.2M project: 74%), and the generated build scripts generated by Netbeans are hard to maintain for users without the IDE. Managing Hudson builds without NetBeans becomes almost impossible.

JavaFX and Maven

We looked for java developers that already tried to merge these 2 technologies, and found out that few actually succeeded. We were looking for a clean integration with minimal POM configuration.

For compiling JavaFX code, we found:
- An outdated Plexus JavaFX compiler plugin that replaced the java compiler. The issue with it is that we could not mix java and javafx code in the same project.
- You can also use the antrun plugin to activate the javafxc Ant task from Maven. The configuration is very verbose and the integration with the Maven model is minimal.
For generating JNLP files, the WebStart plugin is working correctly but it is flattening the references to external jars. It means that all dependent jars are copied in the same folder than the jar of your application. We want to avoid the duplication of jars in the end user WebStart cache, so we liked to use the maven repository layout format for referencing jars.

So, we wrote 2 Maven plugins:

JavaFX Compiler Maven Plugin: compile javafx source files without interfering with the java compiler. It activates the javafxc Ant task using the Maven model to fill the classpath and other parameters.

JNLP Generator Maven Plugin: The plugin will activate Velocity to generate a JNLP file out of the master template and all Maven model data (your project information) and especially maven runtime dependencies. The generated JNLP file will be distributed by Maven automatically.

We used them first on the small demo that depends only on the JavaFX platform, then we used them to "Mavenize" the TwitterFX code base.
So, here is the source code for the Maven version of TwitterFX. In this check out there are no jars in Subversion, no Ant build script, and no classpath. There was 500 lines of Ant code (XML and properties files, managed and generated by NetBeans), that became 200 lines in the POM file (and can be a lot smaller). The main difference is that the POM file is readable and maintainable.
To build it, you'll need java 6, Maven 2.0.9 or above (2.1 recommended) and to run "mvn -Pjfrog install".
We still have one issue today: It is not possible to create a Maven AND JavaFX project inside NetBeans (NetBeans: Maven + JavaFX project? - The Java Posse | Google Groups).

Deploying the Application

In a modified version of Artifactory we are managing WebStart requests toward JNLP and jar files. This extension provides codebase modification of JNLP file attribute based on the server settings, and on-the-fly signing of the main jar and its dependencies. If you deployed a JNLP applet you can even preview it directly from Artifactory!

Using the Maven 2 repository layout to reference third party jars allows the end user WebStart cache to reuse the dependent jars URL. To avoid users re-downloading common jars between javafx examples, the javafx.com demo files are all uploaded at the root of this server!

Re: Contrasting Artifactory and Nexus

2009-01-25T12:35:00.009+02:00

I do not care too much about product comparisons coming from product vendors since they are usually biased and written with a single mindset of "how do I make my product look better". Being labeled with an "I'm nonobjective" sticker from the start, I tend to take them with limited trust.

I recently came across a blog post from Brian Fox of Sonatype, which ships Nexus Pro, a competing product to Artifactory, trying to contrast the two products. We, at JFrog, actually like and thrive from this competition as it is giving us a lot of motivation and directions on how to become even better. However, the comparison done in this blog is actually not a very useful one. Not only it is written with the prime goal of looking for cons in Artifactory, but unfortunately (or, actually, fortunately for Artifactory) almost all points brought up range from plain wrong to just pure FUD.

Let's look at each point at its turn:

1. Network: WebDAV vs. REST
The author suggests Artifactory uses WebDAV with Maven. This is plain wrong! I don't know where this "fact" was taken from, but it is just not true - Artifactory exposes a pure HTTP/REST interface (GET/PUT) which is used by the HTTP wagon of Maven.
Maybe this poor assessment came from the fact that Artifactory also offers WebDAV clients support, but this is an extra feature, on top of HTTP/REST support.
So, assuming that Artifactory will be slower, like the author did (because they tried WebDAV in the past and found it slow) is completely irrelevant.
BTW, while we're at it, we have found the Maven WebDAV wagon to perform much better with huge file deployments and to consume considerably less memory on the client side.

2. Storage: Relational Database vs. Filesystem
For some reason, the author is tossing away the robust architecture used by other, much more widely used, repositories, such as: SVN, YUM, APT-GET to name a few.
Magical tales about "re-healing" of lost metadata do not work when this metadata is not reproducible from pure file data (like file name and path). Metadata has to travel with the data as one unit. If I want to know who originally deployed an artifact, there is no way I can restore this piece of information from the file name.
There is more to it, like when you delete and artifact, Artifactory makes sure that both you deletion, the maven-metadata and the indexes are updated as one transaction - no need to schedule jobs or manually fix annoying inconsistencies that will break your build later on.
The Artifactory 2 storage is rock solid today, and we also fully support incremental backups (including incremental metadata). A disk error could make your complete storage become unreadable (at least without complex recovery efforts, LVM anyone?) and I would like to meet in person a CM who doesn't take backups. Sorry, but saying that your data is at risk because you are using a database store is nothing but FUD.

3. Storage Size
This might be true and might be attributed to several reasons, like keeping richer indexes. Anyway, unless you are running on a 3 years old machine, then frankly, who cares? Really, would your criteria for selecting one product above the other would be because one uses a couple of few gigs less storage space? If yes, I suggest you check the hard drives available today on both server and consumer machines. Same goes for consuming>64MB of RAM (though we do run Artifactory with the default VM heap). Come on...

4. Nexus Doesn’t Interfere
By default, Artifactory takes a conservative approach towards repository pollution, and is trying to shield you from some of the chaos that is going around the Maven central repository, mainly with older legacy artifacts. Most of the wrong POM specifications can be found duplicated on repo1 under the correct coordinates, so Artifactory produces a meaningful error, urging you to correct your POM spec.
Anyway, this protection can be easily switched off by going to the artifactory.system.properties file and commenting out the line that says: artifactory.maven.suppressPomConsistencyChecks=true. So again, this is irrelevant.

Sure, the comparison made by the Sonatype blog is nonobjective. It is natural for a competitor and, of course, I wouldn't expect it to mention pros about Artifactory like simplicity and excellent user experience, integrated UI driven LDAP integration that you don't have to shell out $3,000 for, support for encrypting user passwords in settings.xml, find and remove legacy deployed modules across directories, single deployment for a bundle of artifacts, local repositories import, etc. But, this comparison is also irrelevant, since it relies on wrong or missing information and sometimes on pure FUD.

We do have a lot of appreciation to you at Sonatype for all your contribution around the Maven ecosystem. It's also natural that you want to push your commercial product, but it is very unfortunate when you decide to promote your product by trying to make other products look bad. It's also a shame, that instead of putting valuable time into developing new great features for Artifactory, I have to use this time for clearing up this kind of wrong information. There's a whole community enjoying our efforts, so let's give them good open source products and invest the time in productive directions.

Avoiding Clear Text Passwords in Maven

2009-01-25T01:41:00.018+02:00

On Secure LDAP Integration: Avoiding Clear Text Passwords in Maven Settings and Controlling Login Fallback

First, Some Background
One shortcoming of Maven is that it requires you to store your repository passwords in clear text inside the settings.xml file. Now, this stops being mere annoyance and starts becoming a real security hole once your repository's authentication is done by integrating with an external security service, such as LDAP. SSL can help you overcome transmitting your password in clear text, but storing your LDAP/Active Directory password in clear text is a different story, as it opens the door for anyone who grabs your settings file to all resources accessible by you on the organizational network: email accounts, shared documents, backoffice applications etc. Certainly not a welcome side-effect.

Artifactory includes an out-of-the-box LDAP integration, that can be quite easily set up and tested from the web UI. Soon after introducing LDAP into Artifactrory, we received an inspiring feature request from Patrick Crocker, that led us to implement the current encrypted password support which overcomes this security hole in Maven.

BTW, you can use encrypted passwords even without LDAP, of course, for just not letting people seeing what your clear text passwords look like.

How Does It Work?
First, an Artifactory admin has to enable password encryption by going to Admin:Security:General and setting the password encryption policy to either "Supported" or "Required":

This activates secure password generation for each user based on a secure key stored as part of the user's details.
From now on, any user can use his profile page (your login name on the upper-right corner) to get an auto generated Triple DES secure password to use in his settings.xml file -
just type in your current password and take your secure password for the Encrypted Password field.
Artifactory also gives you a sample server xml element snippet for use your settings.xml.

Controlling Login Fallback
A common case that often appears when integrating with an external authentication system is that you want to force all users to go exclusively through LDAP, except for a couple of special users that need to be able to log in using internal passwords even when the LDAP server is unreachable (e.g., when a change on the LDAP server requires adjusting the connection settings), or when no LDAP authentication details exist (like in the case of a CI build server). For this kind of users, just leave the "Disable Internal Password" flag off in the users' details panel.

That's it!

Enjoy

Artifactory 2.0 has been released!

2009-01-05T22:38:00.003+02:00

We are pleased to announce the availability of Artifactory 2.0.

Artifactory is an advanced Maven repository manager, offering powerful enterprise features, such as LDAP/Active Directory integration and fine-grained permission control, behind an easy-to-use user interface.

With this second major release of Artifactory, Artifactory is 100% configurable via an Ajax web UI and is packed with many enterprise-level features out-of the box:

Built-in UI configurable and UI-testable LDAP integration (other integrations possible via Spring Security).
No more clear text passwords in your Maven settings.xml - Artifactory offers auto-generated encrypted passwords, which is extremely important when using external authentication.
Ability to undeploy complete versions and remove in one go hundreds of old artifacts deployed from the same module.
Sophisticated role-based access control (RBAC), e.g - you can allow only certain groups to see sources.
Security domain level administrators.
Real time view for the effective RBAC on a selected artifact/node.
Ability to deploy a bundle of zipped artifacts and import a Maven local repository into Artifactory.
True undeploy with immediate Maven metadata updates.
Real time tailing of server logs from the UI.
Remote checksum validations.
Built-in support for MySql.
Tomcat install script has been Improved and the ability to run as a Windows Service.
Special support to run under IBM WebSphere.
An artadmin command-line tool for performing various administrative tasks.
The underlying architecture to support high volumes of concurrent downloads and uploads have been updated.
Help on nearly every UI element.
Improved, complete user guide.

Artifactory is available for immediate download from the JFrog Site or directly from SourceForge.

The detailed release information is available from JIRA here.

The Artifactory User Guide can be found here.

You can also try the Online Demo.

Enjoy!

The JFrog Artifactory Team

Lists and Contextual Menus, MMI nightmare!

2008-11-20T15:23:00.000+02:00

I'm more of a "server side" kind of developer.
When I see too much HTML, CSS and Javascript, I need pills!
Still, I like to challenge the MVC architecture of UI frameworks. I even wrote my own web framework for fun (but some poor developers, in India I think, are now suffering from it), and played with Swing developing Stellarium4Java.

So, UI is not my cup of Tea. Now, for the latest release of Artifactory, I needed to managed the communication between the UI Designer and the Web developers. I learned a lot from this interesting experience.
During this process I encountered a really basic UI design issue. It was so basic I got worried I missed something!
The point is: The way we are used to manage lists, selections, actions and contextual menus is totally incoherent.

In real UI applications (fat client and Ajax), you get a list of elements and then you can act on them. Usually, you select a line, then some Action (Buttons, Menus) become enabled and you can activate them. But to minimize the amount of clicks and mouse gestures there are also contextual menus. This is where it gets weird...
You select one line, and then can actually "right click" on another one. That way, the list of Actions on the contextual menus does not match the one in buttons and menus.
You can force select the line where you right click, but few Applications are doing it!

The other issue is that contextual menus are really not that great (i.e. I hate them!):

They are not Web Browser friendly.
They don't actually save much in terms of clicks and gestures (right click, move, left click).
They get in the middle of reading the lists of elements.
They never disappear when you want them to, and always disappear before you find the menu item you want to click on.
To remove them go away you need to click elsewhere, crossing your finger you did not actually click on "buy in one click" button.

One solution on the Web (and on the iPhone/iPod, where there is no right click :), is to add action buttons directly in the list itself as columns. That works and can be nice looking (with good icons) but this technique get very heavy (bunch of garbage icons all over). Anyway, it is not scalable: 5 icons it's already too much.

So, the UI designer came with the really cool idea: A hovering toolbox appearing on the side of the list when rollover the line element.
At first, I thought:

It may work but we'd be going against a big UI muscle memory about how to use a list.
People will hate it!
The toolbox is too far from where my mouse is, and going there will be very annoying.
We are not here to reinvent UI and Men Machine Interaction!

But, anyway I took the chance and we implemented it. To see it, you can go to Artifactory live demo, do a quick search on "log" for example, and hover above the list. It's still a beta version and the previous select/link actions are not removed, so it may be confusing. As anonymous user your action list is limited, but you can still get the feel of it.

I'm using this feature for 2 weeks now and frankly: I love it!
I hover on the list in the left side of it, I can read the line I'm on, and then activate the action with one click. Just natural moves and ONLY ONE click.

May be I'm under the influence (it's our application at the end :), and that's why I'm anxiously waiting for the feedback!
What do you think?