5 Reasons to Keep Software Projects Small

Thoughts of system architecture have been cropping up recently so I wanted to layout some reasons why keeping software projects small is a good thing. A small software project is one that has one main purpose and few features. A small software project can be described to a new person in one sentence: "that's the product catalogue repo, it hosts a public list of products for our clients and has an authenticated admin console." Here's why you should Think Small.

5. Code Ownership

Let's start with the obvious one, maintainability. People, developers included, generally take more care of their own stuff than others'. Software is no different. It's a side effect of keeping projects small, but a very good one: Those projects written by one or two developers will be much better looked after and cared for than projects with multiple owners. I know this because I have worked on small projects written by a few people and large projects written by many. It is less likely for a developer to feel engaged when they are 'in someone else's code'. Motivation is lacking and code craftsmanship suffers when devs are contributing to a project that they do not feel ownership of. Their best work is rarely produced and what does get committed can often contradict other areas of the system in terms of coding style or naming conventions. Over time, unless significant effort is made to enforce standards and motivate developers to take ownership then these changes turn big projects into a Big Ball of Mud. Once that happens, no one wants to work on the project and progress halts. TDD and clean code can help, but they are only preventative measures.

With smaller projects, its much harder to end up in a BBoM situation. There simply isn't that much code in the system. Projects are kept from reaching the point where no one is confident in making changes because the project responsibilities are few and its purpose well defined. You cannot accrue technical debt in a project with no room to take short cuts. And if there are, then they are easy to spot. Its easy to answer questions organisation questions as the need for a clearly named folder structure is minimized.

4. Point Person

That is the go-to-guy when questions are asked or when changes are needed. When there are multiple point-people it usually requires a meeting just to make a decision, or worse, leads to deflection of responsibility. When the one point person knows that they are the only owner, they are driven to have detailed understanding of the system's behaviour (as chances are they have written it). The knowledge should be detailed but easy to pass onto another dev, should the original author move on. Failing that, small systems permit re-writes much more readily that large ones. As for the business, its not the nuances of the particular framework that is valuable, but the role that the system plays instead. This information is more transferable with smaller projects.

3. Quality Requirements

When describing a small system, its much easier to gather the quality requirements. These are things like performance, uptime, security, accessibility, scalability etc. The audience or clients of a small system are usually more focused which lets us make design calls in the code. What is the expected throughput of the url? How soon after event x must the event data be query-able? Will we need to serve content to the other side of the world? Mobile?! When the developer knows the extent of the system's purpose they can make judgements about these kind of questions more easily.

2. (Re)Distribution

This point is about how the system can be accessed. Keeping systems small means that they can be consumed more easily in different formats. Obviously, the most common mechanism of communication is over HTTP. The reason it is so popular is because of the high number of client libraries that support the http request response model with JSON data binding, meaning language interoperability. Http also provides built in functionality for authorisation, caching and security. But the benefits of http are not exclusive to small projects. What small projects are suited to is packaging. A packaged product or library is focussed on doing one thing really well. And when a project has few responsibilities then it can be packaged easily. This becomes a virtuous circle when you consider the motivation for packaging. It makes code shareable and more re-usable. Modern systems are a composition of frameworks and libraries brought together in a particular way to provide unique functionality. Package managing tools permit rapid development by providing access to thousands of small systems that have been packaged, used and tested by others.

1. Innovation

Developers like learning new tricks and using new tools. When what the business is asking for can be delivered in small projects it benefits the business and the developer. If a developer has heard about and new framework or library they would like to try out, then they are going to be twice as motivated to take ownership of the system and enjoy the development experience when they get the chance to try it out. It might take slightly longer as learning occurs, but the development team wins as a result. New knowledge is shared and as such the team becomes more experienced. When some work arrives that has to be out the door by Friday, the team that has the most varied experience are more likely to already know the best tool for the job - and deliver. Aiming for small projects means that a team members acquire new skills more frequently as there are more opportunities to try out something new. Innovation and discovery in IT can be a driver for success in digital industries, thinking small might just give your team the advantage.

Availability Theatre: The Four Nines Comedy

What's absurd about setting 'four nines' (99.99%) as a target for system availability - from a DevOps point of view - is that it completely misses the point of what making and maintaining a system with high availability is all about. After all, when DevOps is given a four nines target what we are really being asked is to ensure that the system possesses the quality component of being available. In order to measure quality in terms of availability, we need to talk about the resilience of the system.

Resilience in a system means that it is not fragile - that it can cope with failure. That means hardware failure, failure within the system and the failure of external systems it might depend on. If we are being asked to build-in high availability, then design choices must be taken to ensure that availability is not at risk because of hardware or software failure. In order to achieve high availability we must instead target building systems tolerant of all forms of failure.

This shift in emphasis from 'always up' to 'resilient' causes a change in the way we architect a system. We make different design choices. We think about exception handling rather than logging. We think about scalability in the application rather than buying a load balancer. We think about caching data rather than taking back ups. We think about automated tests rather than call-out rotas.

Take the world wide web as an example. I would argue that the http protocol has helped deliver so much interoperability because it was designed with the understanding that systems need to cope with failure. That's why there is a sophisticated caching model built in. Sure, http is abstracted at widely usable level, but crucially, it is architected to be loosely coupled. The 404 status code did not come about by accident (!). It is a fundamental acceptance of the fact that sometimes links will be broken. And that a resource that contains a broken link can still be accessed. This is resilience. This ideology is much more honest and practical than pretending that our applications will be up forever.

But this is what we do anyway, right? Any decent DevOps team can translate the four nines requirement into proactive rather than reactive system architecture? Maybe, maybe not. I think it is at least worth pointing out the difference, because here's the comedy in targeting four nines: Lets say you have some system maintenance to perform that would improve the system for its users. Or add some business value, or even decrease the likelihood of system failure, but performing the maintenance will cause some down time that will take you below the 99.99% you promised. Would you perform this upgrade and improve the system? Or would you adhere to your number-in-the-sky target? What is the point of limiting system capability because of some arbitrary number? Why make the business wait until next month when you have another four and a half minutes in which to perform it? Some joke.

Perhaps the reason why management teams favour the four nines is because it is an easily understood unit of measurement. There's no unit of measurement for resilience. However, I do like the idea of a chaos monkey. What we need is a standardised chaos monkey unit of measurement.

I suppose the moral of the story is that managers should try to describe just how important availability is and DevOps should listen carefully. However, DevOps should try to satisfy the availability requirement with emphasis on resilience. Management should be sensitive to these system characteristics and look beyond the statistics. DevOps should never promise to deliver and maintain systems that have a pre-assigned number to the amount of time it will be available. You don't know the future and should not pretend that you do. If you do, you're probably wasted in DevOps.

MVC projects encourage poorly organised domain namespaces

The asp.net MVC framework has been a huge breath of fresh air having usurped Webforms as the standard framework for web applications. Everyone knows the MVC pattern wasn't created by Microsoft, and the current implementation of asp.net MVC has some fundamental differences to that of the original pattern found in SmallTalk. But, it has still provided a more lightweight, transparent, extendible and testable platform to build on then Webforms ever was. And its feels so much better suited to the Web. So, regardless of how much it was influenced by Ruby on Rails, I'm grateful that Microsoft were able to bring MVC under the asp.net umbrella.

With that said, I have a small gripe with with MVC. It's a very small gripe and one that can really only be loosely attributed with it, but its the most obvious place to point the finger for this sake of this article: I think the asp.net framework encourages developers to organise domain namespaces poorly.

First, let me explain what I think a good domain namespace organisation should look like - largely learned from Domain-Driven-Design. Domain namespaces (essentially, folders in the domain project) should expose the purpose of the system. The names should closely and honestly reflect the primary concepts of the domain model. The language of the domain model should be reflected in code - this includes the namespaces the code resides in. If we are designing an application based on a domain model described by the Ubiquitous Language we have unearthed in communication between developers and business experts, then our design must share this language. A well organised namespace should allow the Business experts working on the project to look at the root of the domain and should be able to recognise the key concepts in the names of the root level folders.

In any domain, you will always need some classes as part of your design that do not come directly from the model. These classes commonly come as part of a design pattern being implemented. Their presence makes the communication of the pattern clearer, usually by encapsulating away the details, such as in the Façade Pattern. Such classes usually come with a suffix to describe their purpose, such as: Provider, Factory, Repository, Builder or Manager. It is when domain namespaces become oriented around these classes that are external to the model that organisation has failed. A domain namespace should not describe the purpose of the classes it contains. We should never end up with folders in our domain project called Providers, Factories, Repositories, Builders, or Managers because this is never what our domain is about. Yet this is exactly what MVC encourages.

Because MVC is a Framework (and not a library) it gives us handles to work it. Asp.net MVC is useful largely because it gives us answers to questions like "where do I put a new Controller?" or "where do my Scripts go?". With MVC, you know where to add new Models because there is a folder called Models. It doesn't matter that those (view) models may relate to completely distinct concepts, they can still exist in the same folder - and they're only view models anyway, right? They don't tie directly to the domain model, they're just there to make the presentation layer work.

Right. However, the problem is that I see developers make the mistake of following this convention in the domain project. The model is ignored and folders are created to categorise a sub collection of classes that all happen to be "Factories" or other such classes external to the model. The domain project then fails to be decipherable to a business expert and no longer describes itself with the Ubiquitous Language.

Presumably developers do this because Microsoft gave us the template MVC project as is and because its just easier to put that new Factory in the Factory folder with the others. It'll be easy to find later. But there-in lies the danger: If we are not forced to think hard about how to arrange the namespace then we risk designing something that is not true to the model. If we have to think before we add a class, it makes the choice to add a new class tougher. We will only then be able to authorise the creation of classes that have an appropriate place in the namespace of our domain - keeping the design as close to the model as possible. If we unearth a purpose for a class that doesn't appear in the model (and also isn't an artefact of a design pattern) then we must revisit our model and communicate the idea with our business experts and incorporate the idea in the Ubiquitous Language.

In defence of MVC, it couldn't really work any other way. MVC is a presentation framework - its not supposed to be a domain framework (if there is such a thing?). It has license to organise the way that suits this purpose. It is the developer's responsibility to organise that the domain suits its purpose (according to the model). Just because MVC has a layout that groups together its classes by their purpose, does not mean any other project should. The real issue is to make sure that developers are in tune with the domain and to ensure that it is exposed in the design, and not use the MVC framework as a guiding tool for how to do that.

Even inside the domain, perhaps a layer or two in from the root, there is a place for arranging classes that are external to the model together. If the development team is comfortable that nested inside folders (named according to the model) that it is appropriate to name a folder 'Factories' - then so be it. But I would expect to find such folders a rarity. (Just how many Factories would really be needed in one corner of the domain?).

But maybe there is something to learn from MVC in how we organise the domain? With MVC3 came the concept of Areas. The goal of an area is to provide logical Url Routing and help organise the presentation layer classes and assets according to the routing rules. If you have a number of Areas in your MVC3 project, then the Areas folder starts to look like a well organised domain. All the root folders will speak in terms of the domain model, while sub folders will house Models, Views & Controllers and be named as such. So, if there is anything MVC can do to encourage better domain namespace organisation, it is to include Areas in all template MVC projects.

App Settings are for Variables that Change with the Environment

My opinion is that app settings (or web.config settings) are generally overused. It is usually reasonable for developers to expect that certain values in their code will change in the future. The temptation is to capture all these variables in the app settings. Doing so however, is a mistake.

The only values that belong in application settings are things that will change with the environment, i.e. TEST, STAGE, PRODUCTION.

Everything else, including the things we think (or even know) will change belong directly in source code. I've learned this rule only over time, and I think its reinforced by the YAGNI principle.

Explanation

Sometimes business requirements are delivered in such a way that they barely need translating into a model or domain or even into some syntax. They will usually be very short user stories, and not scored very highly. Examples include:

"As a patient I want a reminder 1 hour before my appointment." 

"As a broker I want the message push interval to be 10 seconds." 

"As a user I want the warning message to read: 'Are you sure you want to delete?'"

The important thing about these stories to notice is the precision they provide. Product owners like precision. They like to know exactly what their product will or won't do. Developers however, like abstraction. They like removing precision and extracting the general rules behind the process. Developers see a rule like '... a reminder 1 hour before my appointment' and take from it the underlying purpose of the story: 'something must happen at a certain time before something else happens'.

There are clearly a number of ways in which sending the reminder could be implemented, depending on the delivery mechanism or format that I won't discuss. But what is clear is that our Product Owner wants the reminder sent 1 hour prior to the appointment. Most developers would look at this requirement and expect it to change. The one certainty about any piece of software is that it will need to change.

The temptation therefore is that the 1 hour (or 60 minutes) should not be hard-coded - if the value is going to change, then lets keep it somewhere where it's easy to change like the app settings, right? Wrong.

There are two reasons for this: Unit Tests and Continuous Delivery.

Unit Tests

The user story in question is business value waiting to be delivered. We want to be absolutely sure that we are delivering that value. If we ship code that doesn't have a passing unit test titled "A_Reminder_Is_Sent_60_minutes_before_appointment" then we haven't succeeded. (Or perhaps closer resembling conceivable objects something like "Reminder_Delivery_Time_Is_60_minutes_before_appointment_time"). We should be sure that no other changes we might make in future will interfere with this piece of logic.

You might argue that we could write a test that reads this value out of our app settings. But then what you would have is an integration test. The reason we want to capture this behaviour in a unit test is to get the answer we want in the shortest feedback loop possible. Also, we don't want to have to maintain a duplicate app settings file in our unit test project.

Continuous Delivery

The practice of automated deployment is becoming more common. As Scott Hanselman says, "If You're Using XCOPY, You're Doing it Wrong. I would take this further and say that if you are still RDC'ing onto the live server to edit a config setting (like changing 60 minute reminders to 120 minute reminders) then you're also doing it wrong. This means the ONLY way to deploy new code (or changed code) is to use an automatic task, and never tamper with the live servers. In a lot of cases developers do not even have access to the live servers.

So now that our app settings file is no longer accessible, it is certainly not easier to change. If we want to change our reminders from 1 hour to 2 hours we have to do so locally, commit and deploy.

Not forgetting our unit test of course - if we made our change in code and ran our tests we should expect a failure. This is important. We have changed the rules of the system and should expect to have to update our unit test accordingly. The new rules are then captured and protect our logic against future unexpected changes.

Environments

So when are App Settings useful? The purpose for app settings in my opinion is for values that change with the target environment. Connection strings are a perfect example of this. In a TEST environment we would expect to talk to a different database than a PRODUCTION environment.

This is where app settings and Config Transformations come in very handy. One rule to decide is this: If your app setting is the same in every target environment, then it doesn't need to be an app setting.

What about Code Reuse?

Keeping values in one place is very useful. But it doesn't mean they belong in the app settings. Globally accessible static classes with Constant field variables is a perfectly acceptable way of maintaining consistency and adhering to the DRY principle. 

When App Settings become a Feature

If we begin to update changing values on a regular basis then the overhead of updating unit tests and deploying can become a burden. At this point we have learned that the ability to update such settings needs to become part of the application. A discussion should be had with the Product Owner over the time spent redeploying versus time invested in providing a new feature to allow users to update values without requiring developer time.

Conclusion

App settings are a tempting place to put variables that we expect to change. However, it is more valuable to 'hard code' such values. This is so that we can make valid assertions against the business rules 'at present'. It is in our interest to break a test if this value changes, and know about it as soon as possible. We also know that we should be deploying changes exactly the same way - whether the changes are in the app settings or source code. This means a change to the app settings file is no more difficult or time consuming than it is to change the source code of our system. Target environment variables are the only valid use-case for app settings. This is to allow such values to vary (transform) between environments. App settings should only exist if the value changes between environments.