Productivity in Drupal’s Development (part 1 of n)
Some of you may recall that a couple of years ago (wow, that long?) I wrote an article on this blog entitled “Artificial Intelligence and Repetitive Tasks in a CMS” describing some of the biggest causes of grief I had when using Drupal (and other online presence/communication creators). I felt it was time for another one of these posts in the same style to critique and give suggestions again. This time, not on Drupal as software, but on the Drupal development process. Also note, I love Drupal and make my living from it, all of this is meant to be constructive.
“Drupal” isn’t Drupal.
If you’ve used Drupal for any length of time you know this is true — namely, the “Drupal” you talk about, use and recommend isn’t the Drupal people go to http://drupal.org and download on their server. I’ve been there before when I made the mistake of touting how wonderful Drupal was, and seen the inevitable conclusion by the person I told this to.
They downloaded the Drupal archive, extracted on their web server, set it up, and came back to me. They thought it was crap. Heck, the out of box experience for Drupal *IS* horrible. Of course I explained that “Oh, no, you want to disable overlay, install admin_menu, install views, grab a theme from here,….” My friend, heard this:
See, the Drupal that people love and enjoy isn’t easy to get. Even though we all tout Drupal as everything but a sugar substitute, getting your hands on that “Drupal” joy is nearly impossible without having done it before. Oh, and, profiles are wonderful, but not all that useful because of how the rest of the development cycle works:
Month 0: Drupal is released
Month 3: Most modules become stable
Month 6-eternity: Views, Chaos Tools and Panels become stable
Month 10: Oh, Drupal has BlahShiney API that is AWESOME!
Month 12: A useful profile is actually released in a beta
Month 14: Useful profile’s author forgets about it due to work/classes.
Month 18: Drupal <next version> Beta/RC is out!
Month 18: “You should wait for/use the beta of Drupal , which has KitchenSink API by instead of using ”
Month 20: Useful profiles for <current version> actually become stable.
Month 22-24: Drupal <next version> released, see Month 0.
I think, bearing in mind a few objections here and there and the odd “but we totally fixed this for real this time, again, honest” comment, everyone can relate to this timeline. There is a simple solution for this; but it would need Dries to backtrack his State of Drupal speech and give up on rapid core overhauls.
You see, Mozilla has had this problem before. Mozilla browsers are simply too big for any one person to handle entirely. Because of that, they are split into a series of modular components with a manager (or maybe several co-managers) overseeing patches and the development cycle for that component. These people are called “drivers” and it’s their job to keep development moving, organize with each other and the higher overall project leaders on how to keep their teams moving in the right direction, and to all freeze their codebases in preparation for major launch events.
Some of that terminology probably feels familiar to those who use Drupal. Now, let’s think of how we could apply this logic to how Drupal develops.
Drupal [Distribution] becomes a stable distribution that has Drupal Core and the top 10 or so most common modules that the community needs to actually launch a good Drupal site. Drupal is a launching pad to great web applications (yeah, yeah; it can be a framework, but let’s focus on the idea of a launching pad distribution that is Drupal’s bread and butter). Drupal core would be split into the required components, with all the optional components separated out, along with these 10 or so critical contrib modules to become this base distribution.
Each of these components has a driver, someone who looks after their component, as well as co-drivers who might only be focused on improvements for a certain version or adding new features. This group of people talk regularly to coordinate efforts and are steered by the distribution’s (Drupal’s) driver (Dries or someone else who manages a specific release).
Minor changes and improvements can be signed off on completely by the component driver without going up the food-chain. Major changes or features get pushed to the distribution driver, both for a “yeah, this is a good idea” and a “yeah, this is ready to commit.” This lets key people stay focused on their component and improving it, and keeps being a Drupal release coordinator from turning into a full-time job/life (btw, Angie, you’re awesome to have done that and making it look easy).
This distribution driver sets the timeline for a release, and all the componet drivers also enforce that timeline for their branch. As critical points in the timeline draw close, they will switch new features to a new version, and only focus on improvements for the current (we’re familiar with this now in core).
When version freezes occur, they occur across all components and the result is a coordinated release of Drupal that it actually “Drupal,” the cool experience we want to share and not “Drupal” the “oh, this is crappy” at-first-look framework. Effectively, this means having active “maintainers” with commit authority across all parts of Drupal core. This removes strain from distributions drivers (maintainers) and allows people to be specialized. Also, synchronicity between Drupal core and the most critical modules so Drupal the framework and Drupal the handy distribution people depend on can be both generally expected at predictable times.
This organized and controlled distribution could be considered, to steal from operating systems, ring 0. Everything in this is critical to Drupal Distribution. The next most popular modules in contrib, as well as sponsored projects could form the basis for components in ring 1; less controlled but still given some attention to have ready for releases and security. Ring 2 would be what we all consider the jungle of contrib as it stands now, with only perhaps some reputation changes that folks are already talking about anyways, and I won’t get into.
Feedback and conversation is very welcome, but before you comment please note: I know parts of this have been discussed, and parts of it may even be attempted as due course; so please don’t comment with, “but we’re already doing x part of this, so y and z are invalid.” Also, any assumptions that I know what happens on all of groups.drupal.org or in back-channel developer discussions should be reconsidered before commenting.
P.S.: Remember Mozilla? They became so caught up in looking busy with new version numbers that they started to release minor fixes and improvements as major releases (6 and 7 instead of 5.1, 5.2). Let’s avoid that, shall we?