The Future is Erlang
For quite a while I've been using my spare cycles to chew over a problem. This problem is not one that hasn't been solved before, but one that I feel can be solved in a much better way. From the bit of research that I've done, I can see at least three areas which can be improved on dramatically. So I've decided, after a long period of deliberation, to go ahead and build my solution and release it to the world as a service!
I know, this kind of thing happens thousands of times a day. Some arbitrary geek decided to build the next killer app and expects to make a crapload of cash out of it. While that would be nice, the ultimate goal isn't the cash. The goal for me is to build something that gets used, and for me to begin to master an area of technology that I've had a quiet passion for for a long period of time now.
My plan is simple. While learning the technologies that I will be using I'll be building another application and blogging about it so that all of you can learn with me. Meanwhile, I shall use my learnings to build this other application at the same time.
The final result should be a series of posts which help other people learn the technology stack, give better coverage and visibility to languages and technologies that a lot of the geek world doesn't know about (and really should!), and for me to carve myself a bit of a niche here on the East coast of Ozland. I am hoping to be able to consult my services out to other businesses as a result.
For those that are interested in following along, here is the full tech stack and toolset that I'll be using:
- Erlang - A fantastic cross-platform functional programming language that has some amazing features.
- Webmachine - A HTTP toolkit which sits on top of Mochiweb that makes it easy to build well-behaved HTTP applications.
- Nitrogen - An Erlang-based framework that makes it easy to build websites.
- Riak or CouchDB - Both of these amazing pieces of tech are potential candidates for the back-end storage. I'm not yet sure which one I'll go with.
- jQuery - A great JavaScript library for the front-end.
- VIM - My favourite text editor
- Xubuntu - Ubuntu Karmic Koala with XFCE as the window manager.
- Mercurial - My favourite version control system.
- Trac - A project tracking tool.
- VirtualBox - My choice for virtualisation. I tend to run Windows 7 as my primary OS, so most of my dev will be in a VM. When I get my desktop machine back up and running (with all three monitors!) it'll most likely multiboot Win 7 and Xubuntu.
- Google Wave - I'll be using this for communications with some really switched on people in the Erlang community who have kindly offered to help me with questions and whatnot specific to some of the areas of tech I've mentioned above.
During the course of my blogging/learning I'll be focussing on Webmachine, Riak/CouchDB, Erlang and Nitrogen. All the other tools will probably get mentioned along the way, but the primary goal is to focus on these things.
I'm really excited! I'm currently in the process of documenting my goals and the design for the main application. Once that's done, I'll get started with the fun stuff. I've got a lot to do and it should be a fun ride. I hope you enjoy following along!
Websites are Hard to Build
"It's just a small site, how hard can it be?"
"I thought you could do it for me as a favour. It's not a complicated site."
"This would take me an afternoon, but I don't have time, can you do it for me?"
Have you ever heard these comments before? Have you heard ones that are similar? I've been hearing them a lot in the last few weeks and it's starting to get to me.
I have just had a chat with a mate who is also suffering this pain, and that conversation is what inspired me to write this, the first random rant in a while.
Dealing with the CRUD
One of the things that I think every web developer on the planet is sick of doing is building CRUD methods for their applications. It's the kind of thing that we seem to do as developers over and over again despite the fact that this task is essentially monkey work.
There have been quite a few attempts to nail this issue on the head in various programming camps, and it's these attempts that I'd like to go over - partly because I'm also sick of CRUD, and partly because I don't know enough about all of these solutions to know which one's work best in what scenario (hopefully this is where you guys come to the rescue :)).
I have to make sure that I'm clear - I am not talking about creation of supporting object-graph loading or anything funky like recursive deletion in code (rather than as a DB setting). I'm simply talking CRUD and CRUD alone
1. Code Generation
This is quite a common solution to the problem. One of the developers either builds a tool, suggests a tool, or downloads a tool that can be used to generate a stack of CRUD methods off some form of data definition. Some of these utilities are able to read the database schema directly, some of them need XML definitions, others do neither and require some other form of manual intervention. The good utilities simple provide a basic interface to the database's definition and allow you to wrap your own templates around them.
The thing I don't get with this solution is that there's still a hefty chunk of work to be done to get the templates for the code and the stored procedures ready and bug-free.
2. Hibernation
This is something I know very little about at the moment. From what I can see on the surface, Hibernation refers to the handling of the persistence of objects (eg. in .NET or Java) to and from relational databases such as MS SQL Server or MySQL. It attempts to decouple the database implementation and schema from the system's domain model - which is a good practice that aids us in preventing the domain model from being polluted with details of the underlying data store. Apart from that, the framework handles the mapping of objects to the data model and vice versa - I liiiiike, it is niiiiiice.
This sounds very nice, but I have to reserve judgement as I'm yet to give it a spin myself. If it does what it says on the tin, then we're all in happy joy joy land. I'll try and get round to having a play with it in the next couple of weeks and see how easy it is to get going. If I can, I'll try and benchmark it a bit too so we can get an idea of the performance. I'm interested to know how well this solution scales.
Apparently Microsoft are looking to create something similar to this (or are in the process of creating it). I believe they're calling it "ObjectSpaces", but don't quote me on that
3. Object Databases
I haven't really kept up to speed with the advances in the field of object databases, but I've recently been handed a link to something that looks quite interesting called db4o which looks to be a Java and .NET implementation of an object database. The idea of an object database isn't new, but this framework looks like it's making good ground.
Unfortunately, in this case, it looks like the performance isn't too crash hot and it's not really scaling that well. Not just that, but I'm not sure if the issues of concurrency and data reporting have been nailed either. Again, this is something I'm yet to really play with, and I'll give it a fair go before dismissing it. At the moment my gut feeling is that it's not yet a viable solution for anything more than a hobby project, despite the apparent ease of use. I love the way it's so easy to persist things and retrieve them again, but until the above issues start being nailed there's no way it can be considered for anything of a larger scale.
What are the others
This is a question for you guys? I'm sure there are stacks out there which are worthy of mention, and I just have no idea what they are. Feel free to send over some links to other viable solutions and let me know what you think of them.
EDIT --
I'd just like to add a quick follow-up to this. Christof over at db4o did me a favour and brought to my attention the current purpose of db4o, which is something I didn't really know before (his comment can be read in the comments section). My focus while writing this blog post was to explore the options for the replacment of the issues faced in an "enterprise" system where there are potentially hundreds of thousands (if not millions) of records and a large amount of simultaneous transactions taking place. It looks like, at least for now, db4o isn't intended to be used in such a scenario. Perhaps in the future this is something that will be targetted? Also, when I stated that it looked like it was handy for a hobby project, I wasn't implying that hobbyists are the only people who would find it useful. Clearly it already has a use in embedded environments - I simply meant as a back-end DB for people building systems they may be using at home as part of a hobby project.
I'm sure there's stacks of information on this on their site and users forums, so I should get my butt over there and do a bit more reading
