g :: The future of dbsettings

I took a look at Satchmo recently, and I noticed that its new configuration system (now in Satchmo 0.6) looked vaguely familiar. I had a brief conversation with Bruce about it, and it turns out he forked dbsettings! And I mean that in a good way.

As it turns out, they had issues with their previous configuration system (a mess of settings files, it looks like), and dbsettings looked similar to something they could use. Of course, they had a variety of additional needs, so they did what open source is meant for: they took and made it their own. They took out a few things, added many more, and bundled with Satchmo proper.

Some of the additions Bruce mentioned seem like they could be very useful for a greater variety of projects, so I thought I’d see about pulling them back into dbsettings itself. The more I look into it, the more I see why Bruce just forked the whole thing rather than just submitting patches. There are some fairly fundamental changes going on, and it’s enough to make my head hurt.

In the end, I’d like to see Django have some kind of unified run-time configuration system. Whether that be dbsettings, or Satchmo’s configuration, or even whether it’s in Django itself or just a third-party app, it doesn’t really matter to me. dbsettings started as a simple app for one project, and now it’s becoming something of a maintenance nightmare, as there are so many unique needs that it “should” cover.

As it is, it’s already grown beyond what I personally enjoy. Most of that was due to the prospect of it eventually being included in Django itself. As I mentioned above, I’d love to see some kind of unification, but I trying to get it into core has already required some compromises that I’m sure are for the best, and they certainly led to some of the maintenance trouble.

So, I’ll outline the concerns raised so far, from both dbsettings and Satchmo combined. This probably isn’t all of them, but it should get the majority. Also, note that these are in the order they came out of my head just now, nothing else:

Settings are defined in Python, and referenced directly as Python variables
Values can be changed at run-time, without a server restart
Values are cached until updated, to speed up simple access (the common case)
Settings can be defined either on a class (usually a Model), or in a module’s global namespace
Settings are grouped together into classes that can be instantiated in multiple locations
Editing of settings is limited to qualified personnel
Settings groups (configurations) can build on each other
Settings can be included or excluded based on the values of other settings
…I’m sure I’m missing some

So, I’d like to ask for opinions on the subject. Should dbsettings grow to encompass more use cases, or should I expect people to write their own forks, as Bruce has done? Should (can?) run-time configuration be unified, so that third-party apps can rely on one common framework? Should such a framework be included in Django proper? If not, would it be dbsettings, Satchmo-configuration, or some new app altogether?

Is there anyone else out there interested in this stuff? As my last post showed (despite its markup errors), I’m in the middle of many things, and those are just the ones I expect to release to the community. If this is to happen, I can’t be the only one behind it.

Comments

At 10:07 a.m. on Nov 2, 2007, patrys said ...
If it's generic enough (think newforms versus oldforms) to allow people to build on top of it (create new setting types or override validation and/or visibility callbacks), I vote for inclusion of dbsettings in the contrib module.
At 3:04 p.m. on Nov 2, 2007, Doug Napoleone said ...
I desperately want to sit down with people at a white board and go over this problem.

I believe it can be done, and it should be part of django as a contrib app. It does not need to be the complete solution for all use cases, just the average use case. People often try to use the admin for things it was never intended, and people will do the same for dbsettings. One thing the admin does properly is in how it can be extended. This again does not solve all issues, and it cannot.

You have a very good rough draft of the unified interface, it is the implementation details which cause the most problems. The current implementations (both dbsettings and Satchmo) are essentially storing the data as strings, and co-opting the DB backend code to do the translation. This, I believe, is where some of the complications come in. You loose all the power of the DB and the ORM, and have to reinvent it. It is cool, it is slick, but you then need to do a bunch of funky contenttype stuff at syncdb time to get it all to work. I think a combination of dynamic models and dynamic type relations will go a long way.

There are a million other related things I would love to hash out, but blog comments, and even irc chats are next to useless. The transmission rate is too low. Nothing beats a laptop, whiteboard, colored markers, and a willingness to explore alternatives.
At 3:29 p.m. on Nov 2, 2007, Marty Alchin said ...
Well, there's always PyCon! I'm really trying to work out how to get there next year, so with any luck, we'll be able to get some Django gurus together and discuss it in more detail. I sincerely doubt much will change in this area by March anyway.
At 2:15 a.m. on Nov 3, 2007, Malcolm said ...
The discussion about whether it should go into django.contrib is fairly irrelevant. There's nothing different, functionally, about being a third-party app to being in contrib/. So don't focus on that.

Build the application you want and see if people want to use it. Then, after it's shown it's utility, inclusion in django.contrib becomes a possibility. But it shouldn't be an issue for the design.
At 12:55 p.m. on Nov 3, 2007, Marty Alchin said ...
Malcolm, I unfortunately didn't realize that in the first place. I built what I needed, then released it, and started making changes to please as many people as I could, in hopes of getting it into contrib. Now, I realize that's not the best, but the fact is that it happened.

Speak up!