Productivity Techniques for .Net Application Development
In a software company, competitiveness has a linear co-relation to productivity. The huge variance in productivity at different companies let better companies run circles around slower ones. They innovate more, they are more profitable, and they get more customers.
It is interesting to look at the situation in India; one of the top destinations for IT projects. I have been working here for about a decade now. Since the early days, programmer salaries have seen at least a 4-fold increase. Competent programmers in India are no longer an order of magnitude less expensive than their western counterparts. Indian IT Services companies can no longer afford to throw large teams at problems, ignoring team productivity statistics. At the same time, a modest improvement in productivity can significantly contribute to the bottom line.
But indeed, productivity is a truly global challenge in software engineering. In the context of the .Net development platform, this article looks at using recent technological innovation to dramatically improve productivity in IT projects.
The Big Leap
Enterprises and IT Services companies have predominantly used Java or Microsoft platforms for building Line of Business applications. Within these two platforms, programming idioms have been more or less the same since C# was largely based on Java. While there were improvements in tooling, language capabilities and expressiveness had not changed much. Tooling can only improve productivity to an extent, before hitting the limitations of the programming language.
But the main .Net languages C# and VB.Net made huge leaps over the last couple of years.. So much so that it eclipses everything done till that point. Microsoft borrowed many concepts from Functional Programming and the academia, which suddenly opened up new ways to express logic. The techniques described in this article are based on these language innovations.
Three Pillars of productivity
1. Declarative Programming
I will borrow Wikipedia’s definition of Declarative Programming:
Declarative Programming is a programming paradigm that expresses the logic of a computation without describing its control flow. Many languages applying this style attempt to minimize or eliminate side effects by describing what the program should accomplish, rather than describing how to go about accomplishing it.
Here is an example:
Example 1: Imperative (typical)var collection = new List<int> { 1, 2, 3, 4 }; var oddNumbers = new List<int>(); foreach (var num in collection) { if (num % 2 != 0) oddNumbers.Add(num); } |
Example 2: Declarativevar collection = new List<int> { 1, 2, 3, 4 }; var oddNumbers = collection .Where(num => num % 2 != 0); |
In Example 1 (left), the task of finding odd numbers is accomplished by looping through the collection and adding matching entries to another list. But Example 2 simply defines what an odd number is. That is, it should not be divisible by two!
The difference is very fundamental. When applied to a business function, an imperative approach specifies how to execute the business function while a declarative approach merely defines the business function.
Consider some real business logic, written in imperative style:
//Get Customers who deposited money in 2010;
public IEnumerable<Customer> GetCustomers_With_RecentDeposits()
{
var customers = new List<Customer>();
var conn = new SqlConnection(CONN_STR);
var cmd = new SqlCommand(“SELECT * FROM Customers C JOIN Deposits D ON “ +
“D.CustomerId = C.Id WHERE D.Year = 2010”, conn);
//Lengthy code to retrieve the dataset and create customers omitted.
return customers;
}
//Get Indians who deposited money in 2010;
public IEnumerable<Customer> GetIndianCustomers_With_RecentDeposits()
{
var customers = GetCustomers_With_RecentDeposits();
var indians = new List<Customer>();
foreach (var cust in customers)
{
if (cust.Country == “India”)
indians.Add(cust);
}
return indians;
}
Here is the same example, written in a declarative manner using LINQ
//Get Customers who deposited money in 2010;
public IEnumerable<Customer> GetCustomers_With_RecentDeposits()
{
var customers = from customer in Db.Customers()
where customer.Deposits.Any(d => d.Year == 2010)
select customer;
return customers.ToList();
}
//Get Indians who deposited money in 2010;
public IEnumerable<Customer> GetIndianCustomers_With_RecentDeposits()
{
return GetCustomers_With_RecentDeposits()
.Where(customer => customer.Country == “India”);
}
With the declarative approach, we simply specified what we wanted and left it to the runtime and compiler to decide how to go about doing it.
2. Type-safe, Verifiable Code
One of the biggest benefits of using a LINQ query as in the example above is that they can be verified for correctness at compile time. The LINQ query is a data structure built using the types referenced in the query and can be checked for type-safety. As a result, if a field name or data type was incorrectly specified in the query the compiler would immediately complain and report the mismatch. In contrast, if the SQL string in the imperative approach referred to an incorrect column name or an invalid relationship, the error is only detected when the application is run.
Of course, Type-Safety is not just about queries. All code should be type safe; strings should only be used to in Resource files or in messages displayed to the user.
Here are more examples:
//Avoid the dictionary, use a strongly typed view
ViewData[“Username”] = “jeswin”;
//Strings used for column names!.
customer.FirstName = (string)reader[“FirstName”];
customer.LastName = (string)reader[“LastName”];
customer.Age = (int)reader[“Age”];
Being able to identify bugs at compile time makes a huge difference in productivity. With such code, the programmer is able to change code without worrying about its impact on other parts of the application.
In addition to this, making code analyzable by the compiler through strong typing allows for easier refactoring. For example, renaming a Field in an Entity (used in LINQ queries) is as easy as right-clicking and selecting ‘Rename’. The IDE will automatically change all LINQ queries to use the new Field name.
3. Composable Business Logic
One of the biggest advantages of LINQ is that it makes queries ‘composable’. In other words, simpler queries and business logic can be combined to form more complex queries.
This is better explained with an example. Assume we are writing and banking application and there is a function called GetHighNetWorthCustomers() which lists people with a large account balance. Say we want to find high net worth customers who also have a PREMIUM plan.
If we are not using LINQ for data access, we would probably have two Stored Procedures or SQL Statements which fetch this data from the database. An oversimplified version of this might look like:
High Net-Worth: SELECT * from Customers WHERE Spending > 100,000
High Net-Worth and PREMIUM: SELECT * from Customers WHERE AccountType = ‘PREMIUM’ AND Spending > 100,000
As we see, SQL doesn’t yield to easy re-use. They sit in isolated form, each confined to a separate function. Not re-usable, Not ‘composable’.
public IQueryable<Customer> GetHighNetWorthCustomers() {
return Db.Customers().Where(customer => customer.Balance > 1000000);
}
//High Net-Worth customers with a PREMIUM Plan
public IQueryable<Customer> GetRichPremiumCustomers() {
return GetHighNetWorthCustomers().Where(customer => customer.AccountType == “PREMIUM”);
}
It was straight forward to re-use the existing query which fetched High Net-Worth customers. In a large application, such composability will dramatically improve productivity. Also, in line with the techniques discussed earlier it is expressed declaratively, and is type-safe and verifiable. Not to mention, a lot more concise.
var richIndians = GetRichPremiumCustomers().Where(customer => customer.Country == “India”);
Composability techniques with Extension Methods
There are people who prefer to use extension methods to compose functions. This provides for slightly more concise code. Whether to use this or not is a matter of taste.
public static class CustomerModule
{
public static IQueryable<Customer> HighNetWorth(this IQueryable<Customer> customers) {
return customers.Where(customer => customer.Balance > 1000000);
}
public static IQueryable<Customer> Premium(this IQueryable<Customer> customers) {
return customers.Where(customer => customer.AccountType == “PREMIUM”);
}
public static IQueryable<Customer> Indians(this IQueryable<Customer> customers) {
return customers.Where(customer => customer.Country == “India”);
}
}
This allows us to combine these functions in a very elegant manner:
var richIndians = Db.Customers().HighNetWorth().Premium().Indians();
Refactor-ability
A few months or a few sprints into a project, teams are generally hesitant to fix less optimal code written earlier in the project for fear of introducing bugs and having to re-test the entire application. This should not happen; inefficiencies and bugs multiply exponentially as more code is written on top of the problematic parts.
The reluctance on the part of developers to fix problems can only be avoided if the cost of making changes are manageable. For example, half way through a project splitting the “CustomerName” column in Customers table into “FirstName” and “LastName” is many hours of work since the entire application might have to be retested. With the new approaches we discussed, this is likely to be 30 minutes of work. It is important to keep your codebase Refactoring-friendly and Agile.
Last words
My company provides architectural and technical consulting services to various IT Services companies in India. Our first-hand experience with a variety of projects in these companies tells us that a significant number of problems faced by teams can be eliminated by using the methods discussed in this article. In our opinion, recent changes to .Net languages represent the most significant mainstream language innovation since the advent of object oriented programming many years ago. The question is how quickly companies will adapt and adopt them.
Especially for IT Services companies in India, the future is moving away from offering lower cost talent to high quality engineering expertise.
Jeswin Kumar
AgileHead
AgileHead offers architectural consulting services and training to IT companies in India. You can email us at services@agilehead.com.
What IDE do you plan on using?
enclojure looks more friendly to me, but perhaps that’s only because I am not so familiar with emacs.
enclosure looks very cool.
For those people who really like Eclipse, there is the clojure feature clojure-dev.
Also, you might consider looking at the “D” language for applications that will need to connect to native C/C++ libraries, e.g. desktop apps. Where Clojure feels like “Lisp done right”, D is “C++ done right”.
Hi Jeswin, Clojure and Scala are really nice languages for sure but fall way behind Groovy in terms of industry take up. In a recent survey in Germany, 30% of Java shops are now using Groovy. If you add up use of Scala, JRuby, Jython and Clojure they are in use in only about 10% of Java shops combined. I haven’t done the numbers but I think it is about the same here in Australia. Language choice is a very subjective thing, so I encourage people to choose what they prefer - just providing you with some feedback on your perceptions of the Groovy community. Cheers, Paul.
Hi Jeswin, Clojure and Scala are really nice languages for sure but fall way behind Groovy in terms of industry take up. In a recent survey in Germany, 30% of Java shops are now using Groovy. If you add up use of Scala, JRuby, Jython and Clojure they are in use in only about 10% of Java shops combined. I haven?t done the numbers but I think it is about the same here in Australia. Language choice is a very subjective thing, so I encourage people to choose what they prefer - just providing you with some feedback on your perceptions of the Groovy community. Cheers, Paul.
Lots of smart hackers are already using Clojure.
I’m puzzled.
A few years ago, you took a risky bet and decided to base your company on an obscure and potentially dangerous technology.
That project failed for exactly the reasons you suspected when you made that decision.
And now you want to make the same mistake by basing your company on yet another bleeding edge technology?
If seems to me you are more interested in experimenting with fun software stuff than building a sound business.
As for the hundred year language, I?m betting that C# or Java is probably in a better position to achieve that goal than any of the other languages you listed in this post.
Good luck, though, I admire your bravado :-)
@Steve
Just like most other failures, ours had little to do with our choice of technology. I was suggesting that the choice of technology is a small company (or a small group of programmers), serves to drum up interest and enthusiasm.
As long as the interface remains the same and the app is fast, I don’t think users will care. And as I was saying, those Mono services never needed restarts ? so Mono really served us well as far as the technology is concerned.
Nitpick:
It’s generally referred to as “Lisp”, not “LISP”.
Cobol will be the 100 year language first, it’s already 50….
:D
Strictly speaking, both Fortran and Lisp are older then Cobol….
you can still stick with Mono though. Boo and Nemrle deserves your attention !
Lisp will not win!
(Too (many) pa(r) ent (hese) s)
There has to be a reason why Lisp didn’t pick up since when it was introduced…. readability.
Parens indicate in literature (a side thought) or something that can be skipped.
It is just not natural to express 2+2 as (+(2,2)) or whatever, in code.
IMHO, it will forever remain a cult language. Something like Sanskrit, Latin etc - good to know and helpful to understand the finer things in (coding (can’t resist)) life!
And what about F# ?
> Also, you might consider looking at the “D” language for applications that will need to connect to native C/C++ libraries, e.g. desktop apps. Where Clojure feels like “Lisp done right”, D is “C++ done right”.
To be precise - check out “D programming language” & “TANGO” (the framework).