OJ’s rants What would OJ do?

13Apr/0913

I Don’t Need Your Frickin’ Framework!

Scaffold FailHow many companies have you worked with/for that have their own framework? How many have been in the process of developing their own framework? How many have been in the process of redeveloping their own framework? How many have taken another framework and hacked improved it?

When I attempt to answer the questions listed above, I start to shiver. I feel the need to burn my clothes and take a very long shower.

In the early days of my career, frameworks were few and far between. These days it is no longer the case. An alarming number of companies are carrying the burden of their own frameworks. A huge number of private and public sector organisations are currently paying people to write new frameworks! For some reason, many naïve project managers and team leads feel the need to invest a great deal of time and moolah into building the next-generation, latest-and-greatest, all-singing-all-dancing, cover-every-possible-web-and-rich-client-case framework before they start to do what they're being paid to do: solve a business problem.

The last job I worked on was one of those places. On my first day I was told that the project team (which at that point was just 2 of the 6 developers) were building a new framework. They were also building a code generator that used this framework. This dynamic duo of programs would be used in conjunction with each other to build any kind of application under the sun. It would solve all of the issues without the developer having to think, all while making poverty history.

What a joke. The code generator was never finished and the framework was never used. The money that was intended to be burned while adding more features to an existing application disappeared in a blur of Greenfield over-engineering of a framework that simply was not required.

This kind of problem happens so much these days, and it's just appalling. What I want to say is: I am frickin' tired of it!

The example I mentioned above is probably the worst I've seen for a while. I'd love to go into a fair bit of depth to uncover some "interesting" features of the framework, but it's not worth it. I would like to highlight just a few points about it though:

  • It was based on CSLA, which is already a rather large framework that attempts to do way too much via an enormous amount of code. The author decided that some parts of CSLA weren't good enough and hence needed to be "improved". The "improvements" were atrocious.
  • It was supposed to be Open Source, but as far as I'm aware it never made it onto the Web, hence I can't point you at it.
  • In some areas of code, you were required to invoke functions through a generic MessagePortal helper object which invoked the required function via reflection after attempting to infer a stack of unnecessary information. This was interesting, as you knew what you needed to call. But you still had to pass in function names as string parameters, hence losing knowledge, and wait for the framework to regain that knowledge by inspecting the code at runtime. Even though the excessive use of reflection was <sarcasm>an obvious win</sarcasm>, we also copped the added bonus of having another 35 (yes, that's right 35) function calls appearing in our call-stack.
  • Those developers who attempted to use it were bitten constantly by esoteric and undocumented pitfalls in the design.
  • It took around 3 times longer during development to do the "basics" (such as create objects and have them load from a DB). This is compared to using no framework and loading objects manually via ADO.
  • It was unable to load any form of object graph.
  • It not only leaked database connections every time you made a database call, it leaked two extra connections during start-up while it attempts to read some database information, such as schema, into memory.
  • The code generator was written in such a way that the first half of the generator would be used to generate the other half (epic fail)!
  • The author wasn't able to use it.

I hence conclude that this framework was a large, unwieldy, bloated, over-engineered, slow and riddled with design flaws. This is all for something that was actually totally unnecessary. Why? ...

.. because this was "The Framework" v2.0! Yup, it was the second version. The first was also a dog's breakfast, but at least it was finished.

So what's my point? Am I trying to say that use of any framework is bad? Having helper libraries and well-designed platforms to build on are an epic fail?

Most definitely not. What I am saying is this: there are people out there with minds immeasurably superior to ours who have already done a better job of solving this problem. Stop writing your own frameworks. Stop writing code generators. There are already options out there.

So to all of you wannabe framework developers: stop what you're doing. You're wasting cash and not adding value. We've already got a framework. It's called Microsoft .NET. Now get out of the way and let me use it!

  • OJ
    @Ryan: Thanks for the insightful comment.

    You can rest assured that none of my judgements are made at speed. Particularly when it comes to technology. I have had a look at CSLA, as I have previously mentioned. I have looked deep enough to be soundly disturbed. Yes, there is a Data Portal (which is what the Message Portal) was built on top of. In my years as a software developer I have never ever seen something like that make it into production. In my opinion, it is for good reason.

    Yes, I do know enough about CSLA to know that even the most simple mechanisms are made quite complicated. Your argument suggests that having 30+ frames added to a call stack for a simple function call is acceptable given that the rest of CSLA does the same. In my view, the fact that it's common or atypical is another indication that CSLA is overengineered and bloated. There's no excuse for having a call stack that deep to make a simple function call. Yes, I know it's there "in case you need to do stuff", but it's another example of YAGNI, and a very bloated and slow one at that.

    To answer your other question, I most certainly did try to influence what was going on. Unfortunately, due to the nature of the management structure and the positions that certain ignorant people were in, I was unable to make any changed. Keef is indeed right, he knows me very well. I am vocal about my views and am quite happy to have discussions about this kind of stuff with those involved. I had attempted to highglight the issues with the framework, but I was simply ignored and basically shut down. Ignorance is bliss isn't it? So I was in the same boat as you described. As a contractor who came in rather late to the project I wasn't in any position to make the decision. I did, however, do my best to influence it. My best wasn't good enough.

    That last part is quite interesting! You're right, there does seem to be a fear out there over being tied to a framework that isn't supported or owned. If you use the example in my post, you can't hardly blame them! The framework was based on the open source CSLA. It was hacked to pieces by an inexeperienced developer who made it worse than it was in the first place. This is quite common of open source forks.

    Don't get me wrong, I love open source, and I see great value it in. Unfortunately, it seems to be abused on a fairly regular basis and people end up creating thousands of bastardised branches of code that they should have left alone.

    As far as picking a framework and sticking to it goes, I do agree that it's stupid that people are scared to take it on. NIH really is an issue, particularly in the world of bespoke software development.

    @Keef: Thanks mate, you're right! I only gave up when my contract ended.
  • Knowing OJ he'll have ranted about the problems with the framework to deaf ears till the cows came home.
  • Ryan
    I agree that less frameworks around the place are better than not. Your right, in my experience almost every organisation that I have worked for has had a framework that they are developing and/or have developed. Each of which are used to varying degrees of effectiveness.

    In the case you have described I personally wouldn't be so quick to judge. For instance, you mentioned CSLA, if you have a look at CSLA the concept of the messageportal is a core part of CSLA (except that it is called a data portal) and uses a lot of reflection. Also you mention the stack being fairly large (I think you mentioned 30+), again if you know CSLA you would know that this sort of stack is not a-typical depending on the scenario you have.

    From the sounds of what you have described, did you try and influence what was going on and make them aware of the problems (i.e. the database connections, the object graph, etc). When I am faced with a problem like this, I usually try and help show them what is wrong with what they are doing and that reinventing the wheel isn't helpful - lets assess what’s really going on here. In some cases, I am not in a position to make the decision, but I always feel I should put my 2c worth in at the time and not just after the fact when it fails. But I am sure this is what you have done and who knows what else was going on at the time.

    Also in my experience the decisions surrounding whether to build a framework or not usually comes down to "some" businesses being unwilling to have a hard dependency on open source software. Which I think is for the most part completely absurd. I think what we as an industry are lacking in a general standard by which open source projects can be rated so that when we go into bat and try and convince the people who don't want to have a "dependency" on open source projects, we can say that this project is better than another one, not just because I or you like it but because the community has given it the following - quality, maintained, reliability, etc rating.
  • brodie
    First and foremost, nice post OJ, and I have to agree that these days it doesn't make much sense to roll your own framework for the run of the mill business applications, in the same sense that it doesn't make sense to write your own bug tracking tool.

    I've been around long enough to witness the dubious ethos of some developers (with tendency towards contractors ;) that is to deliberately create an overly complicated solution with the sole purpose of fulfilling their coding fantasies, extending their own contract, and fattening their wallets ... and this can be excruciatingly frustrating.

    From a business point of view you are right ... but the one thing that concerns me is that you may be stomping a little on creativity and innovation.  I have to agree with @tarn and his comment about the people who 'show us a better way to roll', people with the engineering spirit and can't help themselves but to pull things apart, see how they work, and make them better. Yes, they may make mistakes along the way (at the expense of other people's time and money) but that is how we learn isn't it?

    I think every good developer probably needs to cut their teeth on their version of a framework, a search engine, or write their own blog from scratch, to help with the learning process and to appreciate what goes into making a decent solution.  I wouldn't want you to send the wrong message and create an army of brain-washed code monkeys who only have a hammer in their toolbox and make every problem look like a nail.

    Hmmm, although, an army of brain-washed code monkeys could actually come in handy sometimes.

    Keep up the good work!
  • OJ
    @tarn: Cheers for the comment mate ;)

    The bane about posts like this is that you have to make sweeping generalisations. To be honest, I really love the line you pointed out, though it is rather extreme!

    There will always be some need for frameworks. I have indeed used ASP.NET Web Forms as well as ASP.NET MVC and WinForms too. They can all be branded as frameworks. Same goes for W?F. Am I happy to use those? Of course I am.

    RoR is indeed genius (though made much better thanks to Merb). Django is not something that I've played with but I have heard good things about it.

    The crux of my issue is this: stop writing bespoke EA frameworks. They're an epic fail :)
  • Great post! I see you've moved on from infamous source of "don't lock type objects"!

    My only issue with it is: 

    "We’ve already got a framework. It’s called Microsoft .NET".

    Have you used ASP.NET Web Forms? 

    There is some awesome stuff those RoR and Django kids are up to. Regular expressions to match request to handlers, active record, templating engines..  And even crazier stuff like multiple forms tags on a web page!

    Shouts to the people who break the paradigms and show us a better way to roll!
  • OJ
    @TheNapoleon: Wow, that's probably the best comment you've ever written :)

    Great stuff mate. It's good to see an insight from someone who has been on the "let's make a framework" bandwagon but saw the light!

    I think every geek would love the idea of working on a new framework from the day-to-day work perspective. As you said, it's fun, creative, etc. The problem is that those geeks are too focussed on themselves and not the software. They certainly don't think about the ROI for the business.

    But why? was it a bad design? a bad implementation? was it rushed? No, it just wasn’t needed! After numerous discussions, arguments, and finger pointing the reality was that we just took the wrong path. We didn’t listen to the business need and we tried to solve problems which weren’t really problems.

    Fantastic :) It's awesome to see that you recognised that. It's also great to see that now you're working in a different way you're actually doing a better job of solving problems for the business and keeping the costs way down.

    Love ya work mate!
  • TheNapoleon
    My days of thinking frameworks are over.


    To be honest I once was part of building a framework to build a web site for a business. I thought it was a good idea, and it was exciting! This was programmer heaven. Each team member creativity was thriving, not only our current problems had to be solved, future ones had to be thought of so that we would could fix them too! Oh the joy of working hard and implementing such a clever thing. I couldn't help but fall in love with it, after all this was our baby, something to impress other "geeks" with.


    Ahem, after spending a lot of cash, this framework didn't really help, whilst it still made it to production, this wasn't really a success. Implementing new functionality would create a lot of regression testing, adding extra "products" took far more time and effort than what was expected... and more. But why? was it a bad design? a bad implementation? was it rushed? No, it just wasn't needed! After numerous discussions, arguments, and finger pointing the reality was that we just took the wrong path. We didn't listen to the business need and we tried to solve problems which weren't really problems. Unfortunately we were too committed, and spent too much to be able to stand back and realise our mistake. But hey, who would go and see the boss and say: "Ooops".


    Anyway, forward a few years and I have learnt my lesson. Our current project doesn't have any framework (beside the MS one). Instead we have followed SRP, convention over configuration, simple solutions, no bullshit (why would a business spend 100k to solve a problem which occurs when planets align?). The total spending was 12M to setup the company, employ permanent staff, buy hardware, find a location in the city, build a customer facing website, internal systems, document generation, document scanning, email integration, and more... Better still, give us 10 days and we can ship a new product to our system all with document generation and business rules. 


    Last week, we found out that one of our competitor went live with a very similar business model. Their website has a very similar workflow. We learnt that they built a framework with 30 devs and 15 testers, total cost: 30M. 


    Do I still think a framework would do better? NO FUCKING WAY (pardon my french).



    How would you avoid this pitfall? do SCRUM, keep it real.
  • OJ
    @Michael: You ask some interesting questions. Most of which are very hard to answer given the state of the developers in the industry and the level of visibility the client gets during the development process.

    How many Developers are, some rightly, trying to accelerate their careers through doing this kind of work?
    The short answer is: too many. I don't think that writing frameworks to accelerate your career is the right motivation at all. The previous job I was on fitted this description. The guy in charge of making the decisions wanted to put his new framework in place. This wasn't for the benefit of the client, it was because it was his baby and he wanted it to see the light of day. He burnt the project budget doing that instead of actually fixing the issues requested by the client.

    It's easy to justify that kind of action by saying things like "the previous framework didn't support some of the features that were requested", but I'd argue that this is bullshit most of the time. Fix the old framework, don't reinvent a new one every time the client wants some features that you didn't plan for when you wrote the first one.

    Accelerate your career by doing the right thing for your client. Remember it's not about you it's about the software.

    Is there sufficient knowledge in the marketplace to stop this kind of stuff?
    No, I really don't think so. This comes down to visibility most of the time. Does the client have enough visibility of the decisions and the work that is going on? No. Do they have a big enough say in the implementation? Sometimes yes, sometimes no. What are the causes of this? Abuse of trust. Abuse of position. Unjustified faith in people in positions of power. Lack of understanding of technology. There are so many reasons. Right now we're in a place where developers can take the piss quite freely. It's the good ones which do the job properly for the benefit of the client, not for the benefit of their own CV or ego.

    Who is checking the work / frame work? testing it piece by piece?
    Most of the time, nobody is. This is again due to the reasons stated above. In short, there's no visibility of it. The idea of having a testing team in most places is just non-existent anyway. This is another epic fail.

    How are we measuring for performance?
    This is the million dollar question. Who do you know if a programmer/software developer is any good? Lines of code? Speed of delivery? Accuracy of estimations? Test coverage? Lack of bugs? It's really quite hard to come up with a set of meaningful metrics. Do I know the answer to this problem? I'm not sure I do. I need to think on it a bit more.

    When are we as a community going to stand up together and say enough is enough?
    When the community stands up and has the balls to admit that it's a problem in the first place. When management and clients demand that there be better visibility and traceability of requirements through to work done. When developers stop trying to float their own boats and solve the problem in the best and most cost effective way.

    So probably not for a very long time.

    Cheers ;)
  • Aaaaaaaahhhhhhhh I bet that felt like a good poo.

    The point about adding value is an interesting one.


    How many Developers are, some rightly, trying to accelerate their careers through doing this kind of work?
    Is there sufficient knowledge in the marketplace to stop this kind of stuff; probably not. because developers can convince the Project Manager or Stakeholder easily to implement this frivolity.
    Who is checking the work / frame work? testing it piece by piece?
    How are we measuring for performance? there needs to be some kind of standard that allows those in charge to say "this is rubbish, you are wasting your time, our time and our money"

    The standard of business practice in software development from experience is woeful, the fact this practice is going on doesn't surprise me. When are we as a community going to stand up together and say enough is enough? be able to look those elements who do waste time and money in the eye and be able unequivocally to state they are not meeting the standard.

    Tell me ollie when?
  • OJ
    Thanks guys. It's a rant I've felt the need to share for quite a while but decided to wait until a certain period after leaving the previous joint to vent. I could have said so much more about the example framework, but to me it'd be the same as flogging a dead horse.

    Anyone else had similar experiences?

    @Keef: That's very true re: reinventing. But it's not the only time it happens. I have to add that the same framework I mentioned had a half-baked attempted at per-property security (err... wtf?).. which is ironic because CAS already does that if you want to, and it's built into the .NET framework.

    Epic fail :) Ciao!
  • Most of the content of The Daily WTF comes from people reinventing the frameworks and standard libraries.
    P.S.: Forgot to say good post!
  • Paul Eastabrook
    Too true m8!  Well said!
blog comments powered by Disqus