The Era Of Micro Packages

While the concept of package management in software development is almost as old as the concept of software itself, over the last few years, we’ve seen the rise of what we might call “Micro Packages”. These are packages that are incredibly small in size and scope. A good example of this would be libraries like requires-port, which does nothing besides check to see if a protocol requires a given port. At 38 lines long (including carriage returns and line feeds), it is the essence of a Micro Package. Another more well-known example would be left-pad.

Surely you have not forgotten LeftPad.

How did we get from Mavin and Nuget with entire frameworks as packages to a world where no line of code is too small or insignificant to be a package unto itself? And, more importantly, is this a good thing?

Thank You, npm

First off, this article is written from the perspective of npm. Obviously not everyone uses npm. Perhaps those who are not immersed in JavaScript on the daily will find this article irrelevant to their experience.

npm, in conjunction with the phenomenon that is Github, have together created a platform that virtually removes the barrier to publishing a package. One needs only to login and publish. Two terminal commands and a developer worlds away can download your package for getting a users home directory, so that nobody ever has to write that code ever again.

npm enables this and the community blesses it. Consider this AMA from prolific open source developer Sindre Sorhus which is all about 1 line modules. You read that right. One of the most prolific developers in open-source has a portfolio of 1 line modules on npm.

If the platform supports it and the community signs off, can we consider “micro packages” the way of the future?

I’m not entirely sure. There are some serious drawbacks and concerns.

The Bad Thing About Dependencies Is That They Are Dependencies

Dependencies are a trade-off. You sacrifice some amount of control in the name of productivity. When the developer of LeftPad decided to remove his package, it famously broke Babel and many other enormous and widely used projects. Brought down by a module with 11 lines of code.

However, that didn’t affect any production builds and I would argue that any control that you think you have in life is an illusion anyway. It’s all for naught when your server takes a dirt nap, or your own code just fails. Because 100% test coverage is a lie that we tell Product Managers so they will go back to their offices and get consumed by the sadness that is Microsoft Project.

Still, consider what Micro Packages have the ability to do just by their sheer numbers. The Angular CLI (based on the Ember CLI) pulls down 966 packages from NPM. For a website. I remember when the best thing about the web was that you didn’t need any packages. Now you need nearly a thousand.

Meh – you could argue that number is arbitrary. What do I care what’s in my node_modules folder? The documentation tells me not to mess with it anyway.

For one, there is the issue of speed. It takes my Mac, with an i7 quad core processor and 16 gigs of ram on a 50 gb connection 1 minute 37 seconds and 59 milliseconds to create a new Angular project. I’ve seen .NET projects with tens of thousands of lines of code that could be compiled in a fraction of that time. On much worse hardware.

Which even prompted Sindre himself to once proclaim…

The LeftPad fiasco led some to question whether or not we had completely forgotten how to program. In his article “NPM & left-pad: Have We Forgotten How To Program“, David Haney takes issue with the concept of Micro Packages proposing that a single function does not a package make.

“Functions are too small to make into a package and dependency. Pure functions don’t have cohesion; they are random snippets of code and nothing more. Who really wants a “cosine” dependency? We’d all really like a “trigonometry” dependency instead which encompasses many “tricky” functions that we don’t want to have to write ourselves.”

This argument makes sense to me. At some point, modularity is increasing complexity without the correlation with increased productivity.

However, I can see the other side of this coin. What if I want cosine without having to get all of trigonometry? And why stop at trigonometry? Who’s making the call on whether or not to just have a Math package that does it all? Where is the programming regulatory board who decides which packages are too big and which ones are too small?

I think that David’s point is well made and well taken. Packages should be carefully, not whimsically constructed and published. That’s not to say that one line modules aren’t appropriate, but rather that some deep reflection should be made before publishing such a package to determine whether or not the granularity makes sense. Would it be better to publish a package that only pads left, or would it be better to submit your feature as a pull request to a larger and more comprehensive string/integer manipulation library.

Micro Packages And Commercial Software

Of course, what’s good for the open-source community eventually affects all areas of the technology landscape. We could say that open source is the Paris of our industry. It may take some time for enterprises to realize that nobody is tight-rolling their jeans anymore, but eventually, they will take notice and Micro Packages will begin to infiltrate their infrastructure as well. Historically speaking, enterprises and their firewalls make it rather hard to use any external dependency manager. With the recent move to cloud-based computing, services, and infrastructure, the enterprise is learning to stop all that tight-rolling.

We could say it’s already happening.

When we decided to re-write Kendo UI specifically for Angular 2 (or just Angular, you know what I mean), we knew that we wanted to distribute it via npm. The web development community had spoken. Knowing that we were going to be distrbuting Kendo UI via NPM, the next question we had to ask was “What is the best way to package and distribute Kendo UI?”

Kendo UI And Micro Packages

Keep in mind that Kendo UI is a widely used enterprise UI library. Historically, Kendo UI has been delivered in a variety of ways.

  1. As one monolithic JavaScript and CSS library CDN reference
  2. A download with all of the different modular files (but the styles were still all in one file)
  3. An online tool to customize the download so that you only included the files that you needed
  4. A secure Nuget endpoint
  5. Bower

But consider that even the Kendo UI Button was dependent upon Kendo UI Core (for two-way binding) and therefore it’s modularity was limited even then.

Kendo UI For Angular fully leverages npm and scoped packages to provide the Button component. However, our packages are less Micro and more along the lines of what David Haney suggested with the Trigonometry package. When you run npm install -S @progress/kendo-angular-buttons, you get all of the Kendo UI Buttons available, which would be the Button, SplitButton, ButtonGroup and DropDownButton (Toolbar in development). At this point, we’re basically making a scoping decision for you in terms of what groupings and package sizes we think make the most sense. This provides the required granularity but also doesn’t force one to download 5 different types of Buttons.

How Small Is Too Small?

That is the question and I think the answer is “your call”. If you want to make one line modules and the community finds this helpful, have at it. I’m only interested in solving problems. Micro Packages are the realization that any software that solves problems is valuable, no matter how small.

Related resources: