Last week, Jean-Jacques Dubray published an article on InfoQ regarding Microsoft’s recent announcement of Oslo, a strategy designed to “…take composite applications to the mainstream.” Rather than revolving around a single product, Oslo sets strategic direction for Visual Studio, BizTalk, the .Net Framework, Microsoft System Center and a new product called BizTalk Services. On a side note: Arnon Rotem-Gal-Oz hopes that this final project’s association with BizTalk is more about branding than actual product similarity, a sentiment I share.

After posting this article, Jean-Jacques sent me an email and asked me to share some of my thoughts on Olso for a follow-up article to be published at InfoQ. I sent my thoughts along on Friday, but thought that I would post them here as well.

When we developed a long-term strategy for Composite Applications at my organization, it was obvious that while Microsoft technologies would have a major role to play in many areas of our future-state architecture, there were several vital pieces missing in the Microsoft stack that we would likely need to find elsewhere. I’ve always felt that we weren’t alone in that sentiment, and the Oslo announcement suggests that Microsoft is also well aware of the gaps in their current offerings. While products like Dynamics CRM and MOSS 2007 offer composition scenarios which I regularly point to as examples of end-user and/ or business analyst composition, Microsoft has long been missing the technologies to unify these experiences under a common framework. Though missing in the products themselves, the Composite Applications vision is one that I have seen preached by Microsoft Architects and blogger’s like Mike Walker and others who seem to have a good grasp on the long-term potential of composite applications. The good news about the Oslo announcement is that those individuals are no  longer in the minority. With Oslo, I believe Microsoft has unveiled merely the beginning of a unification strategy that enables composite applications. I believe that this bodes well for clients and non-clients of Microsoft alike.

That being said, There are two reasons why I’m a bit skeptical about the Oslo announcement: For starters, I believe that Microsoft’s stated vision for Composite Applications is too narrow. While the Software + Services and SOA visions are needed, I believe that the end goal of any Composite Applications strategy should be to gradually enable composition up the stack toward the end-user. This is done first by providing a SOA which enables true service and process composition, then by extending those principles to developers of customer applications, business analysts and, ultimately, end users. Oslo speaks well to the former, but the latter is auspiciously missing. I actually don’t believe that such a goal is absent in the halls of Microsoft, but I do believe that it hasn’t permeated across the organization and thus, isn’t given a place in the conversation yet.

The second reason I am hesitant to praise Microsoft for the Oslo vision is because their announcement is related to technologies which are anywhere from 1 year to 3 years or more away from release. Most of the tool updates are two releases away. Microsoft is correct when they say in their press releases that 21st century business is moving faster than IT can deliver, but that statement is true today. Organizations need solutions today, not announcements of solutions coming tomorrow. My organization, for example, cannot wait for a repository to manage models, metadata and services (one of the gaps we knew about in our strategy) when our ability to manage all three is already beyond our control. I honestly believe that Microsoft’s vision for Oslo is a good one, but they are just now announcing plans to provide functionality that most organizations already know they need, which puts them at a disadvantage with those organizations. I can see a day in my company where many of the pieces in the Oslo stack make their way into our architecture, but today we need to keep moving. Of course, the good news is that when SOA and composite applications are done right, vendor lock-in is reduced and organizations can focus on delivering for the business today instead of waiting for the remaining puzzle pieces to fall in place tomorrow.

Back in September, I published a couple of excerpts from an internal paper I wrote on Composite Applications (you can read them here, here and here). At the end of my second post, (which I probably should have broken up into 2-3 posts at least) I discussed the four types of Application Composers. If you missed that section, (which would be proof that I should have broken them up) here’s a recap:

1) Business Service Developers - These are IT developers focused on providing value-added common business services to all customer solutions teams. Their technical depth is high and a CAF targeted at these users would be similar to what is provided by SOA today.

2) Customer Solution Developers - These are IT developers focused on creating customer-centric solutions by leveraging software infrastructure. Their technical depth is moderate to high and a CAF targeted at these users would need to abstract away service creation and assembly.

3) Business Analysts - These are customer consultants focused on helping customers determine which business needs are best met with technology. Their technical depth is moderate and a CAF targeted at these users would draw many features from current BPM platforms.

4) End Users - These are the users of the solutions created by customer solutions teams. Their technical depth is low and a CAF targeted at these users would need to abstract away nearly all of the technical aspects of application creation and should provide very intuitive context- and metadata-driven methods for application assembly and customization.

While these still make sense to me, I think I completely missed number five on this list. It’s a bit of a wildcard, and it may not apply to every organization, but I think it will apply to more and more organization in the coming years. Here it is:

5) External Developers - These are developers who reside outside of one’s own IT organization, but who have development expertise that they wish to leverage to create value-added services that benefit your organization. Their primary interest is in consuming available organizational data and recombining this data with external data or services to create new composites not offered or envisioned by the organization itself.

Now I think that the reason I missed this was because I was thinking internal only when laying out a strategy for Composite Applications in my organization. However, my fearless leader and I have been talking at length about creating a framework by which certain subsets of our information (the right information, of course) could be made available to anyone with the wherewithal to create useful services that we never thought of. Thus, our framework for Composite Applications now has another persona to enable.

So what do you think? Is this a valid addition, or did I have it encompassed in another one of the four? Furthermore, is just one more enough? That fifth category encompasses a ton of people, so do I need another for the technically savvy end user who doesn’t write code, but who screams at creating inventive Yahoo! Pipes applications. I suspect that #4 could represent this individual with a slight modification, but what do you think?

Saw this post from Brady Forest on O’Reilly Radar, which points to the official announcement on the AWS blog. This may not mean as much to many of you who have been using S3 for a while now, but it’s big news for a guy like me because I know better than to bring technologies like S3, EC2 and Mechanical Turk into any serious internal conversation unless I can say “yes” to that SLA question. Looks like AWS is getting ready for prime time. And why not? Most of us have been expecting this for a while now.

I will echo Arthur’s request for more to the SLA, specifically for stats on uptime and availability. I think that all services should provide information like this to be as clear as possible in letting you know what you are getting by consuming said service (and our internal guidelines say as much), but this kind of information is especially vital with services in the cloud. I’m confident that they’ll get there, but in the meantime, I’ll echo that feature request.

On another note, why the heck is O’Reilly charging $149 for a glorified whitepaper on developing applications for Facebook? I use Facebook. I like Facebook. I think there is something to be said about developing applications for it… in certain situations. But could we just take a moment to breathe here before they hype train carries us all away?

I published v1 of my Composite Applications Framework document last Thursday and am pretty pleased with the result. As with all things, it will evolve and improve over time, but I think it was a decent first salvo. I am indebted to Mike Walker and Todd Biske for their input. My favorite part about blogging is being able to have access to and input from such brilliant people.

As I worked back through the draft of the document, I realized that some key visuals were missing in relation to the various composition terms I wrote about in my last post, as well as a view of the CAF architecture. I don’t have time to go into these in detail, but I’ll post them here. Feel free to refer back to my previous two posts for more context. As always, I’d love any feedback that anyone is willing to throw my way.

This first image was designed to depict the relationship between some of the composition terms I used in my last post.

The second image expands on the first, but includes example tiers, containers and assets. Many of these examples are specific to the Microsoft universe, specifically MOSS 2007, while others are generic.

The final image depicts many of the technologies that I believe play a key role in a CAF. This is not meant to be an end-all-be-all list, or even the best depiction available. Just food for thought.

So there you have it.

On a personal note, I’m off for a two week computer-less vacation this afternoon, so the blog will be silent during that time. I’ll be back in the swing of things on September 22nd and hopefully I won’t be so buried in the blogs I read that I’ll have a moment to write a thing or two that weekend.

• The Importance of Mentor - Joe Says: “I say if you want to do something with your life and are willing to make a life-changing and risky change in your life to pursue that goal, then stop being such a chicken and go for it.” Agreed. Great post Joe.

• SOA and the ROI - If you sell SOA without ties to MDM, Portals or Process Improvement initiatives, you’re just selling a technology with the ability for reuse, but no framework to foster reuse.

• Periodic Table of Visualization Methods - Just about every conceivable way to visualize information is on this table, with examples. A great resource to keep around for a visual learner such as I.

If you’ve been following me on twitter, you’d probably be able to venture a guess as to why I’ve been mostly silent for the last two weeks: I’ve been frantically working away at an internal strategy and vision for a Composite Application Framework (CAF). I finished a draft Wednesday and am going through a self-imposed peer review process before publishing next week. In addition to internal peer review, I wanted to start engaging some peers in the blogging world for their opinions on the ins and outs of both the need for and the creation of a CAF in an enterprise. Part of that includes, of course, some discussion around the meaning of “Composite Applications” and CAF, which has several meanings: from being synonymous with SOA (only the beginning IMO) to Web 2.0 in the Enterprise (or Enterprise 2.0 if you like, which I do not). However, before I get into the weeds of terminology, I wanted to share an excerpt from the document that establishes the need for composition period, be it via SOA or Web/Enterprise 2.0. I’ve taken out anything specific to my company, but the rest is true to the first draft of the doc. I would love to have feedback from Mike Walker, James McGovern, Todd Biske, Nick Malick, Mike Kavis and anyone else who wants to weigh in. Going forward, I plan to post some additional sections from the doc for your consumption, review and feedback of any kind. I hope it at least generates some good discussion. So without further ado…

The Need for Composition

To understand the value that a CAF provides for an organization, it is important to first understand environmental trends which are driving IT organizations to reevaluate their current delivery models and to pursue composition strategies. In particular, I believe that there are five factors which illustrate the need for an application composition strategy. These factors are:  

1. The Cost of Building Applications
2. The Problem with Processes
3. The “Consumerization of IT”
4. The Need for Agility
5. The Desire for a Single User Experience 

The Cost of Building Applications

A current reality many organizations face is that IT costs are increasing while businesses are asking for more controls on IT spending. While the high cost of IT solutions isn’t new, more and more organizations are pointing to the cost of integration as a major culprit behind IT cost overruns. According to Ebstrategy.com, “The cost of integration is rapidly becoming a significant barrier to the deployment of new business applications. EBS estimated that in large corporations, approximately 40% of the cost of new applications is spent linking disparate business processes and applications rather than delivering new business functions.” (E-Business Strategies, Inc.) Furthermore, Dr. Jean-Jaques Dubray of Attachmate argues that increasing integration costs are a sign that IT is nearing or has crossed the point of “negative ROI,” beyond which the costs, risks and complexity of integration make it nearly impossible for the organization to realize any ROI from its efforts. (2005, p. 9) The figure above illustrates this theory of increased cost and decreased value as the number of systems in an organization increases.

As a result of this increased cost, Dubray concludes that “…companies are limited in their ability to improve existing business units operations or expand their businesses because all changes in a company’s operation translate into modifying or creating new [Line of Business (LOB)] systems.” (2005, p. 9) The increased cost of applications due to integration cost and the resulting limitation in IT’s ability to deliver value to the organization contributes a great deal to the perception that IT is too expensive and slow to respond to the needs of the business.

In response to increased cost and this perception, Dubray argues that IT ”…can no longer afford to build assets that cannot be reused in future contexts.” (2005, p. 9) Instead, IT must view everything it creates for the business as a collection of discrete and reusable assets. In addition, IT must create a framework that enables both the creation of reusable assets and the reuse of those assets in the creation of solutions.

The Problem with Processes

The systems that IT creates are typically designed to manage discrete steps in a structured process. An example is web content creation, which typically involves story creation, moderation, publishing and retirement. Historically, IT systems have focused on automating approval, publishing and retirement and have been successful at creating value along these steps. However, the largest step in this process (in terms of elapsed time) is usually the first: story creation. It is also likely the most collaborative and would likely consist of several “ad-hoc” process steps where the author collaborates with other individuals to check facts, obtain more information and even elicit feedback on the document in progress. But the system in question assumes that a completed draft is step 1 in the process. The result is that most applications “…do not enable rich collaboration across functional boundaries. This usually leads information workers to use personal productivity tools to perform the complex interactions required to conduct business. However, this in turn leads to a loss in productivity, as users are forced to cross from one set of tools to another, often manually moving the data through means such as cut-and-paste.” (Banerjee, 2006, p. 12)

IT cannot focus merely on process automation to create value for the business when the reality is that most processes involve “…much more work between elements than is represented in the initial structured process. This work is often collaborative in nature. Innovation frequently takes place outside structured processes in the collaboration interactions, where imagination predominates, where the creativity of information workers is truly tapped, and where the critical business differentiation occurs.” (Keyser, 2006, p. 3) Instead, IT should seek to create applications that enable and enhance the productivity of information workers along both structured and ad-hoc processes, as illustrated in the Figure above (Banerjee, What are Composite Applications?, 2006).

The “Consumerization of IT”

The “Consumerization of IT” is a term coined by Gartner to describe the effect of popular technology outside of the enterprise on the expectations and needs of knowledge workers within the enterprise. Gartner argues that knowledge workers are learning new and creative ways to use the web for productivity and collaboration and that they will expect to have these methods and tools available to them in the workplace. According to Keyser, “Based on working with Web 2.0, users will increasingly expect to exert more control over their work experiences and to participate in them. They will expect business applications to adjust to the way they work, rather than accept a suboptimal experience.” (Keyser, 2006, p. 2) An example of this shift has been illustrated by a recent poll conducted on Facebook.com by the Gilbane Group (Gilbane, 2007).

In this poll, participants were asked to select from a list of collaboration technologies which they plan to use in their jobs the most over the next two years. The respondents were then broken up into two groups (18-24 year olds and 25-34 year olds) and a visualization of the results can be seen in the figure above. While it’s a bit of a surprise that younger knowledge workers will expect to use social-networking sites and SMS text messaging as a part of their jobs and not merely for play, a key piece of information in this unofficial poll is that many younger workers would prefer not to use email as a collaboration tool.

The implication, in my opinion, is that IT cannot paint users into a corner by offering one prescribed way in which to accomplish a task or achieve a goal when another tool or method might enable users to be far more productive. In reality, the “Consumerization of IT” means that the use of technology and the tools at an information worker’s disposal are becoming a large part of job satisfaction outside the walls of IT. IT organizations should recognize this trend and seek to provide solutions with the flexibility to meet the technology demands of information workers.

The Need for Agility

Many organizations frequently express a desire to obtain the flexibility to quickly respond to meet new opportunities and demands as a “high priority” which they expect IT to enable. (Keyser, 2006, p. 2) As an organization grows, there is an increasing need for IT to provide the organization with:

• A platform for rapid decision-making;
• Tools to help the business sense and respond to changing market conditions;
• The ability to align business needs with current and future IT assets.

To support these business needs, IT organizations should be able to provide applications which are:

• Quick and easy to deploy;
• Easy to modify, customize and extend;
• Aligned with the current and future needs of the business.

All of these factors illustrate the need for IT organizations to provide infrastructure and solutions which enable an organization to respond quickly and easily to change.

The Desire for a Single User Experience

As organizations grow their products and services, they typically outgrow the LOB systems they purchased or built to manage their earlier products and services. Those legacy systems are typically run to support the existing business, while new systems are created in new interfaces and bolted into the organization to run new businesses. The result for most users, who typically work with similar business processes across all products and services, is a small to large number of disparate systems within which they perform their daily work. Over time, the business begins to call for integration and a unified User Experience for knowledge workers in an organizations. Organizations who have been successful at implementing SOA usually lay the groundwork for meeting this need, but the path to a single User Experience includes a framework which brings data, processes and interfaces into a shared and consistent experience for knowledge workers.

It is important to note here that a single User Experience is not a single user interface or a single system. Rather, a single User Experience is offered by a platform and a controlled number of user interfaces which provide familiar interactions to knowledge workers regardless of the medium, be it Office client, web application, mobile device or smart client. 

References

Banerjee, A. (2006). Building Office Business Applications. The Architecture Journal, (10) 12-19.

Banerjee, A. (2006, December). What are Composite Applications? Retrieved August 20, 2007, from MSDN Architecture Center: http://msdn2.microsoft.com/en-us/architecture/bb220803.aspx

Banerjee, A., Moinuddin, M., & Walker, M. J. (2006). Office Business Applications: Building Composite Applications Using the Microsoft Platform. Redmond: Microsoft.

Dubray, D. J.-J. (2005, May). Composite Applications: Value Proposition and Architecture. Retrieved August 20, 2007, from ebMPL.org: http://www.ebpml.org/capp.ppt

E-Business Strategies, Inc. (n.d.). Composite Applications - Frequently Asked Questions. Retrieved August 20, 2007, from EBS Web Site: http://www.ebstrategy.com/selfservice/composite/composite_apps_faq.htm

Gilbane, F. (2007, June 20). More Data on Facebook users and Enterprise 2.0. Retrieved August 20, 2007, from Gilbane Group Blog: http://gilbane.com/blog/2007/06/more_data_on_facebook_users_an.html

Keyser, C. (2006). Composite Applications - The New Paradigm. The Architecture Journal , (10) 2-5.

Platt, M. (2007). Web 2.0 in the Enterprise. The Architecture Journal , (12) 2-6.

• SSIS Team Development Experiences - Another example of database developers being left out in the cold when it comes to first-class development tools and practices. It’s getting better, but we’re not there yet…
• Adaptive Path UXWeek - Simon Guest is at UXWeek and will be blogging about the experience. I hope that the number of EAs only increases in coming years and hope to attend myself in the future.
• Go old school with a notebook - I’m a tried and true notebook guy. There is no and never will be an adequate replacement for pen and paper for me…
• User experience and the analysts - A nice piece of research done by Rosenfeld Media on major Analysts use of certain UX-related terms. The Gartner ones weren’t surprising to me as a Gartner client (though I would imagine that they would argue that the “Consumerization of IT” term is a UX term, thus should be on the list), but it was interesting to see some side-by-side comparisons. More research along these lines would help an organization like mine have some good insight into which firm, if any, should get our money.
• DSLs bringing the end of single language development? I hope so… especially if one is more open with the terms “language” and “development.”
• Are You a Synthesizer? Enterprise Architects have to be… I sure hope I am. If I’m not, I’d like to become a professional fly fisherman…
• Enterprise 2.0 is about building a collaboration platform that is better than e-mail - The reality, as Jason states himself in this article, is that Enterprise collaboration (I won’t use that term in the title) is less about replacing email and more about repositioning email to be used correctly (messaging) and bringing other tools into the enterprise which are better suited for collaboration (like Parlano). (via Joe Lewis)

• Never Email Anyone Over 30 - I’ve seen many enterprises that think that Email + IM = collaboration needs met and I find it pleasantly surprising to see that IM as a collaboration tool is diminishing in the eyes of the younger generation of workers in favor of things like SMS, Wikis and Blogs (Twitter is probably just about in its own category). What that means however is that the enterprises who are just now getting IM in the Enterprise are still behind the curve… so what’s your strategy for embracing the next generation of collaboration tools? Does an Enterprise vendor have to offer it before we’ll buy it?
• Practical Enterprise Architecture - Some good tips on creating valuable Enterprise Architecture.
• What is Enterprise 2.0 - As much as I don’t like the phrase, or “concept versioning” in any form, I like what the idea represents in terms of driving information access, interconnected applications and devices and collaborative and open work.
• SubSonic - This looks to be a very nice ASP.NET scaffolding framework that parallels much of the project start-up scripts that makes Ruby on Rails such a nice environment to work in. (via Ken Scott)
• Should people adapt to computers? - Agreed Scott. I think that people have adapted enough… it’s time to set them free, and I don’t mean by turning their coffee table into a computer either.
• Enterprise Architects versus Business Architects - While I certainly don’t think that the role of EA is to create services within an Enterprise SOA, I do think that EA has a responsibility in handing valuable business process knowledge to an SOA team that then knows which services are needed. Otherwise, services end up being created to wrap whatever database entities a given project needs next. In any case, the key is that EA has a responsibility to be relevant to SOA efforts, not the other way around.
• Enterprise Architecture is like Herding Cats - Don’t know what I could possibly add here…
• Is Serendipity the Heart of the WS-*/REST Debate? Serendipity is probably a large part of the discussion, but I think a key to Nick Gall’s General Principle of Serendipity (”Just as generality of knowledge is the key to serendipitous discovery, generality of purpose is the key to serendipitous (re)use.”) is that one can rarely (even in an enterprise) plan for explicit reuse of a service. As such, you plan for serendipity.

• Daily Lit - Get daily installments of public domain books delivered via email or RSS. I’ve started with Walden and should be done in 124 days… assuming I read these installments daily, of course.
• The Manifesto of the Futurist Programmers - There’s nothing like a thousand words of propaganda to get your blood boiling over vague ideas and foggy platitudes (via CodingHorror).
• Does Visual Studio Rot the Mind? - A great article from Charles Petzold on the ways that Visual Studio might hinder us as programmers. From a User Experience perspective, I think that the broader critique here is that systems which define our way of working instead of conforming to our way of working probably cost us more than we realize.
• Ha Ha, you must be joking - This is what happens when governance isn’t seen as an enabler and a way to support a person’s natural action. Instead, this is governance from an “us vs. them” perspective.
• A First look at IronRuby - Microsoft’s venture into Ruby looks pretty good so far, and the plan to move IronRuby into Rubyforge is worthy of applause. Tactically, I am still trying to wrap my head around this Dynamic Language Runtime (DLR) idea and it’s place on top of the .NET Common Language Runtime (CLR). Is it just a deferred compilation model? If so, is that really dynamic? Maybe I am missing the point, but I am having a hard time finding a good summary of the inner workings of the DLR. If anyone has anything, please send it my way.
• All models are wrong, some models are useful - A true statement, and the extension of this maxim to include communities is correct as well. When we worry less about “correctness” of the community, its makeup and its work and more about gaining value by creating value, we succeed, I think.

Mike Walker responded to my “Is Enterprise Architecture Declarative or Imperative” question with some great thoughts that I’d like to respond to and expand upon here.

For starters, Mike mentioned that he doesn’t really care for the terms declarative or imperative. I suspect, based on the tone in the rest of his post, that his dislike stems from the cold and mechanical way those terms may sound when used to represent people. In addition, I think that some would interpret, as Mike did, the declarative term as implying a command and control structure in the sense that EA is still telling teams “what” to do. I’d be willing to grant Mike that those are reasonable arguments,  but I will also say that the question was never about the choice between declarative or imperative. Rather, the terms ”declarative” and “imperative” represent the ”world view” an EA team adopts when sitting on one side or the other. In my mind, declarative teams foster a collaborative community, while imperative teams concern themselves with frameworks, governance, models and the “ROI of EA.” Not that these things aren’t important, it’s just that community and collaboration are more so (a la ”Individuals and interactions over processes and tools” in the Agile Manifesto).

Beyond the terms used to frame the debate, I am in complete agreement with the remainder of Mike’s post. The key, Mike says, is that EA is about “fostering community, providing the frameworks for decision making and providing assistance on mission critical projects.” I think that the real key word there is community. We know frameworks in EA (too well at times) and we know assistance (though we might know it by the word governance often), but I think we tend to shy away from community because it’s very contrary to most of the hierarchical organizations in which we find ourselves. But it is without that community that EA is “…viewed not only as Ivory Tower but as traffic cops” as Mike stated. With community, EA is able to voice the strategies of the business and speak from it’s unique vantage point, while at the same time empowering and trusting teams to take the expertise and input of EA and bring it to bear on their projects.

That being said, just saying that EA needs to foster a collaborative community is not enough. I have found that the EA/ SA (or Domain Architects as Mike calls them, a term I like quite a bit) interface can also foster confusion and generate conflict in many cases. I think that EA’s and SA’s could both function better in collaborative roles if they remember a few things about themselves and their comrades.

For Enterprise Architects:

1. People deserve the benefit of the doubt.
2. Coding or not, the people you serve (SA’s) are deeper in the code than you are.
3. Seek to serve, not command.
4. Even if you have the final say in a matter, help people understand your reasons and solicit their feedback early and often.
5. Respect their craft.
6. Keep up with technology and don’t lose your edge..

For Solution Architects:

1. People deserve the benefit of the doubt.
2. Learn to appreciate and understand the role of EA in your organization. Learn to appreciate the different perspectives which they bring.
3. If EA is acting “Ivory Tower,” have the courage to tell them.
4. Treat EA as an ally, not a roadblock.

Not necessarily a complete list, but the key is that EA and SA must learn to trust one another in order to foster the kind of collaborate community which Mike, myself and so many others speak of. If you don’t see a community like that taking shape in your organization, get one started. And if yours is floundering, ask yourself what you can be doing differently before you point to others or the woes of your organization as the cause of your troubles.