This article was originally post on the Medium blog by Gary Wisniewski and I am sharing it with you to make sure everyone sees it. The point of this article really needs to hit home with all programmers, that being that there are to many frameworks. And that most of these frameworks bring nothing new to the table. If you just used best practices, learned to use the core languages, you wouldn't need them at all.
I am a very experienced programmer. I am fluent in at least 15 programming languages, and have written well over 1 million lines of code myself. So, programming and programming systems are not new to me.
Since I began, I’ve seen things done thousands of different ways, in dozens of languages. When I first started programming back in the 80’s, I was enthralled by the ability to package and reuse code — to codify a method and assure that it could be plugged into place like a chip on a circuit board. I imagined that, surely, within 20 or 30 years, nobody would be reinventing anything anymore. That there would be a solid core of productive ways to do things, and that the enlightened programmers of the future (now the present) would never have to think about common, tedious tasks, and systems would work flawlessly, like finely tuned machines, without bugs or incompatibilities.
These enlightened programmers would have moved on to bigger things, and using tried and true engineering techniques, would be improving our world, reducing complexity, simplifying and enhancing our lives. Everything would be shared, and everything would be connected, and the lingua franca of software engineering would be cross-platform, language-independent, operating-system independent modules of wonderful, beautiful, well-tested code.
How wrong I was. What an idealistic fool I was.
The Framework Mess
The horrible state of frameworks (and software scaffolding in general) suddenly hit me this week when I embarked on creating a small embedded Java web server for the Raspberry Pi. Don’t worry about why it has to be Java (it does), or why it’s on a Raspberry Pi (there are good reasons). The point is that, though I’ve written quite a bit of Java, I’ve never really developed a small Java web front-end (I’m not much of a web guy really).
So, I looked around. I discovered that there is the Spring Framework, with people saying that Spring Boot is better, and not so crufty. And the Play Framework: new and modern and more lightweight. And the Struts framework. And Apache Axis. And Apache Xerces. And Hibernate. And JDOM. And the Java Applet framework. And Apache Velocity. And Apache ORO, and JAX-WS. And that’s just getting started.
Now, the idea of frameworks is not at all new to me, so this should not have been a surprise.
I did quite a bit of work in Django when I was in the Python world. Django is a framework, sort of. Like Flask, which was one we considered. Then there’s Pyramid, and Bottle, and Pylons, and Morepath, and even “alternative execution realities” like Twisted, like Tornado, and like Eventlet. Many of these require complete rethinking of the entire paradigm you use to structure and build your application.
I wasn’t surprised. I was even pleased that Python seemed to have “better frameworks”. Before that, I had the displeasure of having to occasionally use php. I was using Drupal, but there were countless others. There were frameworks like Symfony, CodeIgniter, CakePHP. There was the Zend Framework, Aura, Yii and superhero-sounding frameworks like Phalcon. So, yes, of course there would be Python frameworks too.
Then there are Javascript frameworks. Even non-programmers have heard of some of these. Like Bootstrap, Angular, Backbone, Meteor, Ember, React, Ample, Dojo, Enyo, jQuery, YUI, Prototype, Pyjamas, Syncfusion, Webix, SproutCore.
There are even frameworks on top of frameworks. Node.js is (arguably) a framework for building asynchronous systems (similar to Twisted). But within the world of Node we have “Node Frameworks” like Hapi.js, Socket.io, Express.js. And, my personal favorite, Mojito (I’ve never used it, but just like the sparkly lime-flavored ideas it brings to mind). But, if you don’t like a cocktail now and then, you can go for Mean.js, Sails.js, or Koa.js. Or Total.js. Node frameworks seem to just love that “dot”.
Let’s snap back to reality for a moment. What do these frameworks do? And what justifies so many of them?
The second question is easier to answer: Nothing justifies so many of them. Period. The only reason (which I’ll explain later) is that, as an industry, we just really do not yet have a clue what we’re doing.
The first question is a bit harder, but I’m going to take a huge risk, and claim that, at a fundamental level, all of these frameworks are designed to serve the following goals:
- Create a basis for code reuse and compatibility among like-minded programmers doing similar tasks, and…
- Relieve the programmer of low-level common tasks so they can focus on higher level application work.
Such noble goals. Admirable. In fact, you’d think that this were well along the path to those earlier-mentioned independent modules of wonderful, beautiful, well-tested code that would simplify our lives.
But, when you start investigating frameworks of almost any kind, you discover that nothing could be further from the truth. Most frameworks require enormous amounts of commitment. Given the noble goal of freeing us from the slavery of detail, you’d think that if you knew one, that a well-trained polyglot programmer would just glide from one to the other effortlessly.
But no. If you know Sympfony, Django will be a complete mystery. Program using Node.js? You’ll have to start from scratch if you want to learn Twisted. You’re an expert in jQuery? Get ready to relearn everything if you switch to Dojo. Even if you’re an expert, you’ll end up gasping at how perversely arcane the configuration, structure, and deployment can be for different frameworks.
The result is an enormous amount of waste. At some level, the number of frameworks, their constant reinvention, proliferation, and flaws are one of the main reasons we are confronted with overly complex systems that rarely work as designed and are far too costly to build and maintain.
How Did This Happen?
We are in this unfortunate place because of some harsh realities that are due to the immaturity of our industry.
First, the world of modern programming is tribal. Rather than clear communication and openness to ideas, programming communities are founded on us-and-them interactions with self-important leaders where something is good if it’s part of your tribe’s way of doing things, and bad if it’s part of somebody else’s. In such environments, cross-pollination and common goals rarely exist between framework communities. Comparisons of frameworks (especially across languages) become more game-like, prone to offensive and defensive information tactics attempting to find the “winner”.
Secondly, and probably most importantly, there are very few good interface designers, and very few trained architects, nor any clear path defined for becoming one. The world of programmers contains far too many self-taught, self-appointed experts, and far too few strictly-defined concepts and well-understood paradigms. As a result, frameworks are designed by people who are reinventing things and discovering things on their own for the first time. Terms like MVC and REST are co-opted to become mainstream terms without a full understanding of what they actually mean. Variation after variation occur until almost any definition becomes nebulous and subject to interpretation.
It would be like two electrical engineers arguing about what a “volt” is. That is the sad state we are in today.
The Future
It’s going to get worse before it gets better. Ultimately, unless we do something to fix this, software systems are going to become more and more complex, and reliability and security will continue to be unpredictable.
The solution requires that we transform the creation of software into a true engineering discipline. Much like mechanical and electrical engineering, common terms and certifications will need to emerge, and it may take decades. No programmer should be able to work on a system unless they have a proven engineering certification, and no programmer should be able to build a framework unless they have an advanced certification in systems architecture. The Atlantic pegged it well in their article “Programmers: Stop Calling Yourselves Engineers”.
Today’s software community will scoff at this idea. “We already have certifications!” they will say. But, certifications today are more like “resume items”. Above all, they are optional, and therefore have little meaning. A doctor should not be able to cut open your skull unless they have the proper training and credentials, and a programmer should not be able to write code that accepts your credit card details without proper training and credentials. It’s just that simple.
Moreover, this is going to take longer than we think.
Early batteries were found dating back to 250BCE, and finally by the 18th Century people were discovering the importance of insulators and conductors. But, it wasn’t until the late 19th Century that Electrical Engineering was well-defined enough to have university courses.
Today, we are an industry of inventors and discoverers. Our frameworks, our languages, our ideas and speculations are more closely allied to the adventurous experiments conducted by people like Ben Franklin and Thomas Edison.
It’s an exciting time to be sure, but I probably won’t live to see that wondrous day I imagined where beautiful, simple, 100% reliable and reusable software systems are the basis of all our efforts. Anyone reading this may not either! But, let’s not fool ourselves into thinking we’ve arrived, when we’re just making the very first steps.
Now, back to that …. grrr… never mind.
Original Blog Post