MyOpenDraft

The challenges for Informatics in developing software for modern multikernel computers

2011-11-07T00:20:00.000-08:00

Abstract

The purpose of this post is to examine the introduction of parallel computing and the challenges of software development for Parallel execution environment. First I will introduce the idea of parallel computing and up next I will present and evaluate the challenges of parallel computing along with their solutions and finally some conclusion will be drawn.

Vertical & Horizontal Development in Computing

The question arise when we are thinking about how the complex scientific problems of the twenty-first century including climate modeling, genomic research and artificial intelligence are testing the limits of the Von Neumann model of sequential processing.

In the past, computer scientists worked on the new approach to extend the power of computers in vertical manner, this means that they were working on producing huge super computers but with recent advances in technology and reducing cost of resources and arrival of multi kernel processing has helped us to think about new ways to solve huge and complex problem in parallel manners.

Introduction to parallel computing

For the most part, along with a host of new research questions that have arisen in the last decade, there remains a significant challenge today. Parallel processing offers the promise of providing the computational speed required to solve important large-scale problems. In fact, parallel processing requires a big shift in how we think to solve the problem.

Regardless of new hardware technologies, we should think about the new approach of developing software systems and also the way we think about our problem and presenting our solution. (Design and Analysis of Computer Algorithms).

Challenges of parallel computing

For the sake of applying the power and flexibility of multi-core processors, we should think about a new approach to breakdown huge problems into smaller elements. A better illustration of parallel processing occurs when a divide and conquer model is used to solve a task.

In this approach the problem is successively partitioned into smaller and smaller parts and sent off to other processors, until each one has only a trivial job to perform. Each processor then completes that trivial operation and returns its result to the processor that sent the task. These processors in turn do a little work and give the results back to the processors that gave them the tasks, and so on, all the way back to the originating processor. In this model there is far more communications between processors.

In the next step, we should think about how to express our program which can be executable in a parallel computing environment. Functional Programming plays a vital role in this area, since it provide programmer to solve their issue in functional manner rather than sequential processing. There are simple principles in functional programming such as avoiding Mutable states, Lambdas, Closures and more importantly declarative paradigm which help programmers to free their mind about concurrency, synchronization, Race Condition and other multi core computation issues.

Although parallel functional programming helps us to represent our program in declarative manner in order to be applicable for parallel execution, but the problem is remain unsolved without thinking about how we can manage data in parallel computing environment.

Industrial Revolution of Data – Age of Big Data

We’re now entering into new age of computing named as “Industrial Revolution of Data”. In fact, the majority of data will be produced automatically by different kinds of machine such as software logs, video cameras, RFID, wireless sensors and so on.

Due to the considerable decrease in cost of computer resources, storing those data is so cheap, so companies tend to collect and store them in huge data warehouse for future when it can be mined for valuable information. The Big Data now comes to play, working with such distributed, huge and complex data would be impossible or better to say inefficient with existing software and databases system.

We should think about other approaches for storing large set of data which is stored in different computers and in the next step effectively mining and executing queries from those sources. Perhaps the biggest game-changer to come along is MapReduce, the parallel programming framework that has gained prominence thanks to its use at web search companies.

The research in parallel computing has had the most success and influence in parallel databases. In fact, instead of breaking out a large problem into smaller element execute by different threads simultaneously, parallel database help us to store, querying and retrieve data from distributed resources over network effectively.

MapReduce as Parallel Programming Framework

MapReduce algorithm is invented by Google to cope with Big Data in their search engine system. In fact, MapReduce is containing two simple primitives function which are available in Lisp and also in other functional languages. The computation include two basic operation, a map operation which execute on input records containing key/value pairs, and then invoking a reduce operation which collect and aggregate all responses from different nodes.

There are many different Implementations in different programming languages which are exist and used in industry for processing large set of data. In fact, most of NoSQL databases use this algorithm for collecting data from different sources in distributed heterogeneous environment. The biggest advantage of MapReduce is that it allows for distributed processing of map and reduction function. In fact, it allows us, to collect and process distributed data stored in different machine simultaneously.

Conclusion

Parallel computing can help us to solve hug complex problem in more efficient way. In order to parallelize our task we should think about different challenges which we cope in developing software for parallel execution environment.

However, we should bear in mind that parallel computing is useful when we are facing with a big problem which can distributed among different computing agents. In addition, we should deeply think about the nature of problem, time as well as limits and costs of Parallel Programming.

The Management 3.0 - Leading Agile Developers, Developing Agile Leaders

2011-02-27T08:24:00.000-08:00

For people who get the message, this book may prove to be as valuable as Darwin’s book On the Origin of Species.

This book introduce and evaluate different management theories and it will teach you how to use them in your agile environment. In fact, it is really a scientific book and recommended for everyone who wants to know more about agile philosophy from the scientific perspective.

This book will discuss about General System theory, Game and Chaos theory as well as Evolution theory. In this book, Jurgen does a great job of explaining the science behind complexity and how Agile management methods have arisen from the need to manage in complex, dynamic, and unpredictable circumstances

Six Thinking Hats

2010-12-19T07:04:00.000-08:00

While facing with a number of choices, we may find it hard to make a decision, or may always approach problems in the same way. Emotional people, for example, may not consider decisions calmly and rationally. Many successful people think from a very rational, positive viewpoint, and this is one reason for their success. Often, though, they fail to look at a problem from an emotional, intuitive, creative or negative viewpoint. By always using a positive approach, they may underestimate possible difficulties - such as resistance to their plans and be under prepared for dealing with further problems.

'Six Thinking Hats' is a valuable technique for increasing the effectiveness of decision making. Created by Edward de Bono, it makes you consider the decision from a number of perspectives, forcing you to add different ways of thinking to your usual approach.

This gives you a fuller view of a situation. As a result, your decision and plan will be ambitious, creative and sensitive to the needs of others. They will be carried out effectively, and you will be prepared for the unexpected. You can use Six Thinking Hats with other people or on your own. With others, it has benefit of blocking the confrontations that happen when people with different thinking styles discuss the same problem.

Each 'Thinking Hat' is a different style of thinking. With the White Thinking Hat you focus on the data available. Look at the information you have, and see what you can learn from it. Look for gaps in your knowledge, and either try to fill them or take account of them. This is where you analyze past trends, and try to work out from historical data what might happen in the future.

Wearing the Red Hat, you look at problems using intuition, instantaneous reactions, and emotion. Also try to think how other people will react emotionally. Try to understand the responses of people who do not fully know your reasoning.

Using Black Hat thinking, look at all the bad points of the decision. Look at it cautiously and defensively. Try to see why it might not work. This is important because it highlights the weak points in a plan. it allows you to eliminate them, alter them, or prepare contingency plans to deal with problems that might arise. Black Hat thinking helps to make your plans tougher and better able to survive difficulties. It can also help you to spot fatal flaws and risks before you start on a course of action.

The Yellow Hat encourages you to think positively. It is the optimistic viewpoint that helps you to see all the benefits of the decision and the value in it. Yellow Hat thinking helps you to keep going when everything looks gloomy and difficult.

The Green Hat stands for creativity. This is where you can develop creative solutions to a problem. It is an unstructured way of thinking, in which there is little criticism of ideas. A whole range of creativity tools can help you here.

The Blue Hat stands for process control. This is the hat worm by people chairing meetings. When running into difficulties because ideas are running dry, they may direct activity into Green Hat thinking. When contingency plans are needed, they will ask for Black Hat thinking, and so on.

Packt GlassFish Security book contest

2010-10-12T14:17:00.000-07:00

About two weeks ago, I just participate in contest for winning Packt GlassFish Security book by Masoud Kalali, and the great news is that today I received an email which is mentioned that I'm one of the lucky winner of this contest.

I'm really happy and lucky for winning this prize and I would like to thank Masoud and Packt for this reward.

If you are interested about this book you can download free chapter "Designing and Developing Secure Java EE Applications" from Packt website. This book cover most important aspect of security in J2EE Application with GlassFish and additionally it will teach you how you can secure your Enterprise Application by using OpenDS and OpenSSO.

If you want to develop secure enterprise application with GlassFish, I would recommend to take look at this book. I'm sure that it will be helpful.

Best Practices for building Large Scale Agile teams

2010-09-27T09:29:00.000-07:00

Building large teams, either traditional or agile, requires careful design. Bigger teams involve more communication, more decisions, more meetings, more documents, and, of course, more politics.

Agile principles of collaboration, simplicity, responsiveness, and minimally sufficient documentation can be applied to building larger teams but the application is far from simple. Building effective, efficient large agile teams takes a concerted effort in the four areas mentioned earlier: organizational design, decision-making design, collaboration/coordination design, and applying agile principles.

Although collaboration, coordination, and knowledge sharing are critical to large projects, the downside of too much communications can be endless meetings and wading through tons of documentation and emails. Too little or poor communications, however, means no one understands the project nor their part in it. So whether we are doing organizational design or collaboration/coordination design, agile teams always balance on the side of "just a little bit less than just enough."

Organizational Design
In general, as projects escalate from large to very large the number of specialty teams will increase and a few members of the specialty teams will be full-time members.

In addition, as size increases, feature teams will be organized into a structure, product area team, capability team, feature team based on the product's component architecture.

Whereas in a traditional large-team hierarchy the teams might be functional (requirements analysts, developers, testers), large agile teams maintain their cross-functional and product orientation. Where very close coordination is required between feature teams (or between certain feature and specialty teams) cross-linking should be used in which one person from each team sits in on key meetings of the other.

Another design criteria is that a large agile team maintain the flattest structure possible (more nodes, less hierarchy).

This organizational structure focuses on the collaboration and coordination between autonomous but linked groups. As the team size increases to encompass several feature and specialty teams, there is a critical combination of team self-organization and self-discipline required. Individuals have responsibilities within a team structure, and teams have responsibilities within an overall project structure.

This network structure isn't a hierarchically controlled one, but neither is it a pure network structure in which all control is delegated to the nodes. The structure might be labeled a "modified network" structure in which a significant amount (but not all) of the power and decision making are distributed to the feature teams.

Collaboration/Coordination Design
In Agile Software Development, Alistair Cockburn discusses various communications modalities and indicates each one's relative effectiveness.

For example, a two-way face-to-face discussion at a white board is more effective than sending a document to someone. However, for large and especially distributed teams, the enemy of effectiveness remains cost. Although it may be very effective to have a face-to-face design discussion at a white board, the cost of doing it when the individuals are located 5,000 miles apart may be prohibitive, although new Web 2.0 collaboration and project management tools are continually reducing these costs.

So the real question is not just effectiveness, but cost-effectiveness for the particular job at hand. What makes communications design so difficult is that it rests on a foundation of relationships of trust, respect, appreciated cultural differences, and shared context.

A team that is "in sync" can get away with lower effectiveness communications modalities because they have a good relationship context for sharing information. Another team distributed across countries in which there is little respect, trust, or multi-cultural understanding will need higher effectiveness communications modalities.

This also means that the collaboration/coordination design will probably change as projects progress and relationships become better established. Communication modalities that might not work early in a project may work fine toward the middle and end.

Two factors in designing collaboration/communications practices and tools for a team are relative effectiveness of the modalities and the foundation of trust, respect, and understanding of different cultural norms.

Decision-Making Design
How is a large agile team different from a large traditional team from an organizational perspective?

• First, a large agile team has a flatter, less hierarchical structure fewer layers of managers.
• Second, to the greatest extent possible, decisions are pushed out to the feature teams or specialty teams.
• Third, feature team members participate in specialty teams to ensure their input is heard and to take part in the decisions.
• Fourth, team decision making, whether project or technical decisions are being made, are accomplished in a participatory fashion.
• Fifth, specialty teams are encouraged to issue guidelines, not fiats, and furthermore (like managers) they are encouraged to make as few decisions as possible.
• Sixth, peer-to-peer (feature team to feature team) interactions and dependency management are encouraged.
• Seventh, the team embraces agile principles.

Knowledge Sharing and Documentation
Documentation and knowledge sharing are two important issues in scaling agile projects. A 6-person team needs different documentation practices than a 100-person team does.

The Agile Manifesto states, "Working products over comprehensive documentation." Large, front-loaded projects that spend months, and even years, gathering requirements, proposing architectures, and designing products are prone to massive failures.

Documents support communication and collaboration, enhance knowledge transfer, preserve historical information, assist ongoing product enhancement, and fulfill regulatory and legal requirements. They are not unimportant, just less important than working versions of the product.

Understanding comes from a combination of documentation and interaction, or conversation, the conversations among people who have domain knowledge. Furthermore, as the complexity of the knowledge to be transferred increases, the ability of documentation alone to convey that knowledge decreases and the need for interaction with knowledgeable people increases dramatically. Documentation provides content (partially), but conversations are necessary to provide the necessary context.

A little research into the field of knowledge management shows that early efforts to document best practices (software development, engineering, or otherwise)—filling reams and reams of web pages with details—has generated marginal benefit. In thinking about knowledge transfer, one must distinguish between transferring explicit knowledge (written down) and tacit knowledge (in someone's head). "

Tacit knowledge cannot be transferred by getting it out of people's heads and onto paper," writes Nancy Dixon in her book Common Knowledge "Tacit knowledge can be transferred by moving the people who have the knowledge around. The reason is that tacit knowledge is not only the facts but the relationships among the facts—that is, how people might combine certain facts to deal with a specific situation."

In summary, documentation should be designed in a lean, barely sufficient manner, both formal (for retention) and informal (temporary), highly visual and visible, viewed as support for collaboration and coordination, and vary considerably by project size and type (regulated environments for example). Ultimately the issue isn't documentation but understanding.

Self-Organizing Teams of Teams
An agile team consists of individuals who interact within a structure of self-organized and self-disciplined rules of engagement. Individuals have a degree of autonomy within this loose structure, and they, in turn, exercise self-discipline to be accountable for results and behave as responsible and thoughtful members of the team.

Larger agile teams, those consisting of multiple feature and specialty teams, operate the same way, individuals are the agents in teams, whereas feature teams are the agents in a larger project.

In building large agile teams, a network replaces the common hierarchical structure, decision making and collaboration must be carefully designed, and discipline reflects rules of engagement across teams.

Team Self-Discipline
Just as individuals have responsibilities to their teams, teams themselves have to be self-disciplined to work within a larger self-organizing framework.

The behaviors required of teams closely parallel those for individuals:
• Accept accountability for project results.
• Engage collaboratively with other feature teams.
• Work within the project self-organizing framework.
• Balance project goals and team goals.

A team that is unwilling to work within the established framework disrupts the work of the larger project in the same way that individuals who are unwilling to work within the team framework do.

Teams have to align their own goals with those of the project. There will always be too much to do and too little time, and the tendency will be to work on one's team goals rather than on project goals.

Process Discipline
We all know the saying "Don't fix things that aren't broken." For larger teams, we need to give this advice a twist: "Don't always fix things that are broken." Although smaller teams are not immune to excessive process, this tendency becomes more prevalent as teams get bigger.

Just as agile teams don't try to anticipate future requirements or design, choosing instead to let them emerge over time, they shouldn't attempt to anticipate every problem and put processes or practices in place to prevent them. It's often cheaper and faster to fix an actual failure than to spend excessive time anticipating failures that may never occur.

references:
- Cockburn, Alistair. Agile Software Development. Boston: Addison-Wesley, 2006.
- Highsmith, Jim. Agile Software Development Ecosystems. Boston: Addison-Wesley, 2002.
- Rueping, Andreas. Agile Documentation: A Pattern Guide to Producing Lightweight Documents for Software Projects. New York: John Wiley & Sons, 2003.
-Dixon, Nancy. Common Knowledge. Boston: Harvard Business School Press, 2000.

Role of UML in Agile & Evolutionary Architecture

2010-09-26T12:52:00.000-07:00

The Unified Modeling Language (UML) is a drawing notation standard, not a methodology. In terms of the standard methodology framework (roles, techniques, standards, milestones, etc.), it is one of the standards for one or two of the roles.

It is a component of a methodology, but not likely to be a very large factor in the project's outcome.The UML standard does not make prescriptions or even recommendations about when, where, and how much you draw.

With respect to the standard, you can draw out the entire design before starting to program, or you can "think a little, draw a little, code a little," a strategy long recommended by object technology experts and supported by Scott Ambler's (2002) book, Agile Modeling.

Discussion of drawing models quickly gets caught up in discussion of architecture and how much design to do when. You will not be surprised to learn that my view is that different people have different preferences and like to think about designs for different amounts of time before starting to program.

Having said that, it is also my view that a design is a "theory," which desperately needs a matching "experiment" to validate it, or, more importantly, reveal its weaknesses. Usually, the design doesn't survive its first encounter with real code. This means that the sooner the design gets tested in implementation, the sooner the designer learns where and how it needs improvement.

Some people have a very short thinking time horizon before they desperately feel the need for an experiment: on the order of five or ten minutes. Others have a medium time horizon, on the order of several hours or a day or two, during which they mentally explore all the pitfalls, weaknesses, and potential change requests they may have to adapt to. Others have quite long thinking time horizons, on the order of a week or two, during which they do the same.

My experience is that when designers spend more than a few weeks working out a (software) design, they usually have passed the optimal trade-off point, where it would have been useful to create an experiment with real code to get some real feedback.

Taking all those together, I feel that most architects take too long before they make their first experiment, but it is not necessary to adopt the view that design can be replaced just by refactoring and attention to XP's "once and only once" rule.
The subjects of UML, drawings, tools, architectural design, and thinking time horizons tend to get confused all together. If you can pull them apart, it frees you up to work in new combinations.

It is, for example, not the case that all drawings are UML drawings nor must architectures be described in UML, or even in drawings, nor must fancy software tools be used to capture either drawings or UML , nor does spending time thinking mean sitting at a tool. Finally, and especially, nor is it the case that sitting in front of a UML drawing tool means that any architectural thinking is going on.

The approach typical on a Agile project is to use the Walking Skeleton and Incremental Re-architecture strategies. An early architecture is sketched, programmed and tested in the first iteration, and completed, extended and evolved over subsequent iterations. UML might or might not be used to think through it and document the result.

References:
- Beck, K. , Extreme Programming Explained: Embrace Change, Addison-Wesley.
-Ambler, S. , Agile Modeling: Effective Practices for Extreme Programming and the Unified Process, John Wiley & Sons.
-Cockburn, A. , Agile Software Development, Addison-Wesley.

Best Practices of Agile Distributed Teams

2010-07-14T03:48:00.000-07:00

It is always more challenging to manage distributed teams. Regardless of development methodology, distributed teams will be less productive than co-located teams. Alignment and communication are harder because distance, time zones, language, and culture prevent a lot of the informal communication flows that occur when team members are located together.

If you are embarking on significant new product development, think long and hard about whether you want to have a distributed team. If being distributed is a constraint that you must accommodate, Agile will give you better visibility so you can correct product issues more quickly.

Best practices for distributed teams:

Each site conducts a local standup in their morning to address immediate issues.

All teams join a daily teleconference standup, ideally scheduled at a common work time for all. A video-conference standup is better.

Each location has a Scrum Master Proxy and a Product Owner Proxy. The proxies synch with their counterparts regularly and learn to guide their local teams and keep them productive.

Team members visit other sites to deepen relationships and information exchange.

Technology can play a role in mitigating some of the challenges of distance. VOIP and webcams can go along way to overcoming cultural awkwardness and maintaining a co-located feel. It is worth the extra effort to get these technologies working. Distributed teams also need to implement a collaboration tool to function as a virtual task board.

Examples include wikis at the low end and more specialized products like Rally Software, VersionOne, Xplanner.org, and Atlassian Jira with the GreenHopper plugin. There are many other tools, and you should be able to find a solution that fits your needs and budget. As an aside, if you have only one remote team member, the Scrum Master can usually support that person and the team can still even use a physical task board.

All About User Stories

2010-07-11T04:54:00.000-07:00

The user story covers three points:
• The user's role
• The goal the user is trying to achieve
• Why the user wants to achieve it

It takes the following form:
As a I want to so that I can .

As an example:
As a customer, I want to be able to compare two products so that I can easily understand the differences in feature and price.

User stories are an effective way to communicate requirements that tie into personas by specifying a role, and to express the desired capability and the justification for adding the capability. The story provides the context for the product designers and engineers to develop creative solutions to meet th user's need.

INVEST
Bill Wake in Extreme Programming Explored (2001) introduces six attributes of a good story that can easily be remembered with the acronym INVEST.

• Independent—Avoid dependencies between stories. Each story should stand alone so they can be re-prioritized in the Product Backlog at any time and not impact either the order or any other stories. You also want to keep the stories granular, so the highest priority elements of a feature get implemented first. This is also described as separating the feature from the embellishments.

• Negotiable—Stories are a starting point for a conversation with development. They are not a command. Many product managers have learned that the proper way to write requirements is to use the word "shall" to indicate unambiguously that the product must perform the described function.

• Valuable—Always demonstrate why the story is worth implementing. If you are struggling to identify why the story is valuable, go back and spend more time investigating the story with your customers. You may have missed an important aspect of the story, or the story may not be worth the development effort and ongoing maintenance.

• Estimable—The story should be small enough and contain enough detail that the development team can estimate the effort. Epics should still be used as placeholders in the Product Backlog to capture important ideas and signal intent. But as the epic moves towards the top of the backlog, it will need to be split into stories so the team can estimate the work.

• Small—A story should represent between a half-day and two weeks of work and should definitely be small enough to fit into an iteration.

• Testable—Acceptance criteria for the story should be able to be tested to ensure the user's need has been met.

Acceptance Testing

Acceptance tests are written on the back of the user story card. The tests are used to let the developer know when the requirement has been met and to confirm that the product works from the user's perspective.

A story is not complete and cannot be assigned to an iteration without acceptance criteria. Acceptance testing is the responsibility of the whole team. As such,acceptance tests can be added at anytime by any team member. The product manager, however, should look to facilitate this process by sitting down with engineering and QA to brainstorm the acceptance criteria.

Acceptance tests capture many of the functional elements and boundary cases of the feature. By communicating functionality via tests, it makes it easy to for the engineer to know when development on the feature is done and for QA to validate that the feature works.

You may also hear these tests referred to as "satisfaction criteria" This refers to the high-level test cases that the feature should fulfill prior to the product manager, developer, and QA person brainstorming the full set of acceptance tests and boundary cases.

Acceptance testing is not a bug hunt in the traditional sense. The unit testing is in place to prevent bugs from accumulating in the code in the first place. Acceptance testing really focuses on whether the requirement was met. Other forms of testing such as stress or nonsensical input testing may also be warranted and would be conducted as an additional step.

Non-Functional Requirements

User stories are best suited for functional requirements. Non-functional requirements typically seem forced when put into the user story format. A story
to communicate that the product should run in multiple browsers would be written:

As a customer, I'd like to be able to use the browser of my choice so I don't have to download a new browser.

It would just be clearer to write what is intended:
The application shall run on Internet Explorer 7.x and higher, Firefox 3.x and higher, and Safari 4.x and higher.

Constraints and performance measurements can also be particularly challenging, because to be accepted they have to wait until the end of not just the iteration, but of the full release. In the example of multi-browser compatibility, all user interface requirements are dependent on this constraint.

Constraint Cards
Mike Cohn, in User Stories Applied (2004, 178–179) lists seven constraints with examples:

1. Performance: Ninety percent of product searches will return results in
less than three seconds.

2. Accuracy: The software will dynamically generate and adjust reorder
points to provide in stock levels of 98 percent for all standard products
while maintaining less than fourteen days' inventory on hand for 95 percent
of all standard products.

3. Portability: The software shall be designed to be ported to Android.

4. Reusability: The graphics rendering engine will be reusable by our other applications.

5. Maintainability: Automated unit tests must be written for all new code and be run after each build.

6. Interoperability: All documents shall be stored in XML.

7. Capacity: The data mart must be able to store one hundred and eighty million transactions (three million per month for five years) and support the real-time analytics tools.

System Quality Cards

Another technique for handling non-functional requirements is the system quality card. As functional requirements focus on "what the system will do," system qualities describe "how well the system will do it."

Below is a real-life example of a system quality requirement for a navigation system that was forced into the user story template.

As a Driver, I would like a faster load time for navigation, so I don't have to wait for it to load.

In addition to sounding unnatural, the story is neither testable nor estimable. It just does not work well as a user story. Instead, a system quality card should be used.

A system quality card has six areas:
1. Name: The name of the quality (must be unique)
2. Scale: What will be measured, including the units of measure such as seconds
3. Meter: How the measurement will be conducted
4. Target: The desired level of performance
5. Constraint: The point at which the performance is unacceptable
6. Benchmark: The current level of performance

Use of user stories is just one way to capture requirements in Agile development. But by no means is it the only way. Many teams succeed just fine by writing straightforward requirement statements and even producing more traditional requirements documents.

Regardless of documentation format, the goal is to maximize conversation about each requirement within the team and have a workable method to prioritize, re-prioritize, and separate features into small enough increments to fit into iterations.

Reference: Agile Excellence™ for Product Managers A Guide to Creating Winning Products with Agile Development Teams By Greg Cohen

Agile for Today Competitive Business Environment

2010-07-11T02:30:00.000-07:00

Why Agile Works

Agile works because it supports the process of software development in four key areas:

1. Empirical process
2. Daily visibility
3. Socialization of information
4. Rapid feedback cycles

Empirical Process

It is easier to understand an empirical process by first looking at a defined process. A defined process is one in which defined inputs generate defined outputs. This is the original manufacturing model. It is repeatable, and it is the model that traditional software development has assumed

In contrast, the empirical process is nonlinear. It requires frequent inspection and adjustment. It is well suited for new product development, which requires research and creativity—both relatively unpredictable activities. Software development, including the product management step of requirements gathering and validation, is best suited to an empirical model.

Software development teams are always creating something new. They do not follow a straight path with a defined output that is the same each time they go through the process. Instead, teams cycle through knowledge creation and problem solving, As team members formulate the solution, new questions emerge and new requirements need to be gathered.

Occasionally the team has to restart. An assumption turns out to be false, such as the availability of complete and accurate data to drive a critical process. Sometimes the environment changes and a new technology emerges that needs to be supported. At times the solution functionally works but the performance is slow.

Further, there is no escaping the QA team, whom you can always count on to identify some good issues that need solving. The list goes on. To accommodate this, the team must frequently check its progress and adjust its plan.

Daily Visibility

Agile improves visibility into the development process with a daily standup meeting. At the meeting, team members let each other know what they are working on and if they need help. Roadblocks are explicitly called out. Because this meeting is daily, issues cannot be ignored. Further, each developer commits to what she will be working on next in front of the team. Through this process, the team develops trust and the ability to be open and honest.

Socialization of Information

Agile processes facilitate the socialization of information. First, as mentioned above, the daily standup drives trust in the team. This creates an environment conducive to sharing news, even if it is discouraging. Secondly, because team members meet daily to share their accomplishments and plans, individual knowledge quickly becomes team knowledge, and the team stays aligned.

Rapid Feedback Cycles

Rapid feedback cycles are embedded in the Agile process. The feedback loop accelerates the team's learning and supports the empirical process needed in software development.

There are three connected cycles: the release, the iteration, and daily standup. The release may be an extended cycle, but the iteration is one to four weeks, and the daily standup provides feedback every twenty-four hours.

Agile feedback loops share much in common with the Plan, Do, Check, and Adjust (PDCA) process popularized by Dr. W. Edwards Deming in his quality control work: the team members formulate a course of action, execute against it, inspect their progress, and adjust their course accordingly.

Why Now?

Interestingly, Agile is not new. In fact it has been around since the 1990s. It is worth asking: Why now? Why has Agile become so popular?

To be a little controversial, I would say it is because of the failure of traditional serial or "waterfall" development and our attempts to control the software development process. Although I know there are many advocates of waterfall, in my experience it just does not work that well.

Further, the Internet and globalization have had profound effects on the software industry, increasing the intensity of competition and the rate of change. If you are still releasing annually or over an 18-month cycle, by the time your next version is available, the market will have moved.

Further, with the advent of Software-as-a-Service, hosted models, hosted download, and self-updating software, releases can occur with much greater frequency. Thus, companies have looked for a better way to develop software. Agile has met the challenge. Even so, best practices for Agile continue to evolve, including the constant challenge of working with distributed teams across multiple time zones.

Reference: Agile Excellence™ for Product Managers A Guide to Creating Winning Products with Agile Development Teams By Greg Cohen

Lean Thinking

2010-04-29T11:43:00.000-07:00

Two pillars of lean

Toyota president Gary Convis:

The Toyota Way can be briefly summarized through the two pillars that support it: Continuous Improvement and Respect for People. Continuous improvement, often called kaizen , defines Toyota's basic approach to doing business. Challenge everything. More important than the actual improvements that individuals contribute, the true value of continuous improvement is in creating an atmosphere of continuous learning and an environment that not only accepts, but actually embraces change. Such an environment can only be created where there is respect for people hence the second pillar of the Toyota Way.

And from Toyota CEO Katsuaki Watanabe:

The Toyota Way has two main pillars: continuous improvement and respect for people. Respect is necessary to work with people. By "people" we mean employees, supply partners, and customers. ...We don't mean just the end customer; on the assembly line the person at the next workstation is also your customer. That leads to teamwork. If you adopt that principle, you'll also keep analyzing what you do in order to see if you're doing things perfectly, so you're not troubling your customer. That nurtures your ability to identify problems, and if you closely observe things, it will lead to kaizen continuous improvement. The root of the Toyota Way is to be dissatisfied with the status quo; you have to ask constantly, "Why are we doing this?"

Lean Goal: Sustainably Deliver Value Fast

Broadly, the global or system goal of lean thinking at Toyota is to go from "concept to cash" or "order to cash" as fast as possible at a sustainable pace to quickly deliver things of value (to the customer and society) in shorter and shorter cycle times of all processes, while still achieving highest quality and morale levels. Toyota strives to reduce cycle times, but not through cutting corners, reducing quality, or at an unsustainable or unsafe pace; rather, by relentless continuous improvement, that requires a company culture of meaningful respect for people in which people feel they have the personal safety to challenge and change the status quo.

Lean Foundation: Lean Thinking Manager-Teachers

One of the most important things is that most new employees first go through several months of education before starting other work. During this period they learn the foundations of lean thinking, they learn to see 'waste' and they do hands-on work in many areas of Toyota.

In this way, new Toyota people...

•    learn to "see the whole"
•    learn to see how lean thinking applies in different domains
•    learn kaizen mindset (continuous improvement)
•    appreciate a core principle in Toyota called Go See and gemba

Go See means people especially managers are expected to "go see with their own eyes" rather than sit behind desks or believe that the truth can be learned only from reports or numbers. It is related to appreciating the importance of gemba going to the physical frontline place of value work where the hands-on value workers are.

From this, we came especially to appreciate that for successful adoption of lean, there are management qualities needed for any meaningful, sustained success the leadership team cannot "phone in" their lean support.

Toyota is one of few companies that seems to demonstrate these qualities; to summarize

•    Long-term philosophy—many in the company are educated in lean thinking through courses and mentoring from manager-teachers.

•    Long-term philosophy—virtually all management, including the executive level, must have a solid understanding of lean principles, have lived them for years, and teach them to others.

•    Long-term philosophy—manager-teachers have cultivated systems thinking and process-improvement problem-solving thinking skills, and they teach it to others. The culture is imbued with the mentality and behavior, "Let's stop and understand the root causes of problems."

Pillar One: Respect for People

Respect for people sounds nebulous, but includes concrete actions and culture within Toyota. They broadly reflect respect for and sensitivity to morale, not making people do wasteful work, real teamwork, mentoring to develop skillful people, humanizing the work and environment, safe and clean environment, and philosophical integrity among the management team.

The 11th agile principle and a theme in Scrum is self-organizing teams (self-directed work teams), supporting this pillar

Pillar Two: Continuous Improvement

Continuous improvement is based on several ideas:

Go See is a principle not found in many management cultures. This principle is described as critical and fundamental. In the internal Toyota Way 2001 it is highlighted as the first factor for success in continuous improvement. Go See shows up repeatedly in Toyota manager quotes, in Toyota culture and habits, in education on the Toyota Way, and in the research done by Japanese analysts of lean thinking. All that said, it is missing from some derivative 'lean' descriptions and so unfortunately some are unaware of its vital role. In a lean-thinking culture, all people, but especially managers including senior managers should not spend all their time in separate offices or meeting rooms, receiving information via reports, computers, management reporting tools, and status meetings.

Rather, to know what is going on and help improve (by eliminating the distortion that comes from indirect information), management should frequently go to the place of real work and see and understand for themselves. This "real front-line place of work" (gemba) does not mean proximity to the building where work happens, nor does it mean going to visit other managers. It implies to be as physically close to the real front-line work as possible—not sitting in an office nearby, but "breathing the same air." 'Work' in lean does not primarily mean the overhead or secondary work of accounting and so on, but the value-adding work that the customer cares about—engineering, designing a car, producing things, delivering customer service.

Kaizen is sometimes translated as simply "continuous improvement" but that confuses it with the broader lean pillar of "continuous improvement" and does not capture the full flavor.

Kaizen is both a personal mindset and a practice. As a mindset, it suggests "My work is to do my work and to improve my work" and "continuously improve for its own sake." More formally as a practice, kaizen implies:

1.    choose and practice techniques the team and/or product group has agreed to try, until they are well understood
2.    experiment until you find a better way
3.    repeat forever

Five Whys (usually written 5 Whys) is a simple and widely used tool used in kaizen. It helps develop problem solving and root cause analysis skills. In response to a problem or defect, a team considers "why?" at least five times.

In Scrum there is a retrospective workshop each iteration. This is an excellent time for a team to try 5 Whys. The important point of 5 Whys is not the technique or the number 5, but that it is part of the "stop and fix" root-cause problem-solving mindset and culture pervasive at Toyota. People are taught to become deep problem solvers; to not live with problems, but to think things through deeply. There is also a connection between Go See and 5 Whys: It is easy for people to guess wrong or weak answers unless they see the facts at the real place of the problem.

Value and Waste
What to improve during kaizen? In lean thinking the answer requires an understanding of value and waste.

Value— The moments of action or thought creating the product that the customer is willing to pay for. In other words, value is defined in the eyes of the external customer. Imagine a customer was observing the work in your office. At what moments would they be willing to reach into their pocket, pull out money, and give it to you?

Waste— All other moments or actions that do not add value but consume resources. Wastes come from overburdened workers, bottlenecks, waiting, handoff, wishful thinking, and information scatter, among many others.

No Final Process

The implication of kaizen and spread knowledge laterally is that there is not a final or correct 'defined' process to follow everywhere that is communicated from a central process group. Kaizen does include learning and mastering working agreements, but they travel and evolve by the spread knowledge laterally model.

People who have the mindset "let's define (or buy) the central process, write it down, and then we should focus on conformance to it" will not be comfortable with lean thinking. To quote the Toyota CEO, "The root of the Toyota Way is to be dissatisfied with the status quo; you have to ask constantly, "Why are we doing this?"

Lean and agile values and the Scrum method are based on the idea of empirical process: there is no fixed or final process or cookbook that people can follow given the reality of dynamically changing systems, and given the goal of continuous improvement. Instead, in Scrum we are left with the hard work of kaizen—to relentlessly, every two-week iteration, inspect and adapt the process and create yet another "two-week process experiment." In Toyota and in Scrum, the idea is to repeat this cycle until retirement.

References:
- Ohno, T., 1988. The Toyota Production System: Beyond Large-scale Production, Productivity Press

- Poppendieck, M., Poppendieck, T., 2006. Implementing Lean Software Development: From Concept to Cash, Addison-Wesley

- Craig Larman and Bas Vodde,

Scaling Lean & Agile Development: Thinking and Organizational Tools for Large-Scale Scrum , 2008, Addison Wesley Professional

Introduction to Behaviour-Driven Development with Groovy

2010-03-09T08:58:00.000-08:00

What's Behaviour-Driven Development ?

Behaviour-Driven Development (BDD) is an evolution in the thinking behind TestDrivenDevelopment and AcceptanceTestDrivenPlanning. It brings together strands from TestDrivenDevelopment and DomainDrivenDesign into an integrated whole, making the relationship between these two powerful approaches to software development more evident.

It aims to help focus development on the delivery of prioritised, verifiable business value by providing a common vocabulary (also referred to as a UbiquitousLanguage) that spans the divide between Business and Technology. It presents a framework of activity based on three core principles:

Business and Technology should refer to the same system in the same way - ItsAllBehaviour
Any system should have an identified, verifiable value to the business - WheresTheBusinessValue
Up-front analysis, design and planning all have a diminishing return - EnoughIsEnough

BDD relies on the use of a very specific (and small) vocabulary to minimise miscommunication and to ensure that everyone – the business, developers, testers, analysts and managers – are not only on the same page but using the same words.

For people familiar with the concept of DomainDrivenDesign, you could consider BDD to be a UbiquitousLanguage for software development. It must be stressed that BDD is a rephrasing of existing good practice, it is not a radically new departure. Its aim is to bring together existing, well-established techniques under a common banner and with a consistent and unambiguous terminology. BDD is very much focused on “Getting the words right” and this focus is intended to produce a vocabulary that is accurate, accessible, descriptive and consistent.

What's Problem ?

Test Driven Development (TDD) has established itself as a useful--and at its best, profound--improvement in the software development process. It delivers a variety of benefits to its practitioners; some of them are related to testing, but the most important benefits are not.

Experienced practitioners, particularly those that have been involved in helping other developers learn the practice, note a life-cycle to TDD's learning and adoption:

The developer starts writing unit tests around their code using a test framework like JUnit or NUnit.
As the body of tests increases the developer begins to enjoy a strongly increased sense of confidence in their work.
At some point the developer has the insight (or is shown) that writing the tests before writing the code, helps them to focus on writing only the code that they need.
The developer also notices that when they return to some code that they haven't seen for a while, the tests serve to document how the code works.
A point of revelation occurs when the developer realises that writing tests in this way helps them to “discover” the API to their code. TDD has now become a design process.
Expertise in TDD begins to dawn at the point where the developer realizes that TDD is about defining behaviour rather than testing.
Behaviour is about the interactions between components of the system and so the use of mocking is fundamental to advanced TDD.

We have observed this progression in many developers, but unfortunately while most, with a little help, find their way to step 4, many miss the big wins found at steps 5, 6 and 7.

We started wondering why this was, and wondering what we could do to help people get to the good stuff more quickly. Looking closely at this evolution in understanding, while step 1 may be considered to be related to testing, the rest really are not. For the remainder of the steps in this process, the test aspect is very much a secondary concern.

The issue at the heart of this learning process is the behaviour of the system. Developers gain confidence in the systems they build when the behaviour of that system is confirmed.

By writing tests in advance of code, the developer defines the behavioural intent of the system that they are developing. Tests that document the behavioural intent of the system are most valuable as documentation of that system. By specifying the behaviour of the code in tests, developers are able to best consider the interface from the perspective of the consumer of the service rather than as the producer – thinking more abstractly about the nature of the solution and so hiding technical detail.

This focus on behaviour ahead of testing is a focus on the real value in TDD, and in our experience is the mark of the genuinely experienced TDD practitioner.

It is not too surprising that it takes apprentice TDD practitioners a while to realize that TDD is not about testing when all of the nomenclature surrounding it is described in terms of testing.

and what's solution ?

The aim of Behaviour Driven Development (BDD) is to address this shortcoming. By using terminology focused on the behavioural aspects of the system rather than testing, BDD attempts to help direct developers towards a focus on the real value to be found in TDD at its most successful.

BDD aims to bridge the gap between the differing views of computer systems held by Business users and Technologists. It is deeply rooted in the success of TDD and is influenced by ideas like Domain Driven Design.

Its focus is on minimizing the hurdles between specification, design, implementation and confirmation of the behaviour of a system. Thus enabling the incremental delivery of business systems, and in particular in allowing teams new to agile development to quickly get up to speed using these extremely productive techniques.

Getting the words right

Behaviour Driven Development (BDD) grew out of a thought experiment based on Neuro Linguistic Programming (NLP) techniques. The idea is that the words you use influence the way you think about something.

As an example, when I was first getting to grips with TDD, I was pairing with an experienced agile coach, writing little test methods, then writing the code, and generally feeling good about life. Then I went ahead and wrote some code without a test. The coach, JR, asked me why I'd written the code. I answered: "we'll need it in a minute", to which JR replied "yes, we might". By using the word "might", he introduced the possibility that we might not. As it turned out, we didn't. - DanNorth

A coach introducing Test Driven Development (TDD) to sceptical (i.e. most) developers invariably runs into a familiar set of objections.

From programmers:

Why do I need to write tests? That's what testers do.
Writing all these tests slows me down, it's a waste of time.
I'm a good programmer! I don't need to write tests to prove my code works

And from testers:

Why are you getting programmers to write tests? We all know they can't – that's why you need testers.
Are you trying to take our jobs away?
You obviously don't understand testing or you wouldn't be asking programmers to write tests!

This last comment is particularly ironic. In fact, the testers take on a central and very important role in BDD.

The behaviour-centric vocabulary helps us to avoid these common misunderstandings and focus on BDD as a design and delivery process.

Concentrating on finding the right words led us to think about the process differently, for example, the fact that the words we use in BDD are very much focussed on the behaviour of the system led us to better understand the very close relationship between the stories we use to specify behaviour and the specifications we implement in BDD in place of tests. It also helped us gain further insight into some of the failure modes we have experienced in TDD projects.

The BDD Process Model

A SubjectMatterExpert (typically a business user) works with a BusinessAnalyst to identify a business requirement. This is expressed as a story using the following template:

       As a Role
       I request a Feature
       To gain a Benefit

The speaker, who holds the Role, is the person who will gain the Benefit from the requested Feature.

This can also be paraphrased variously as ...

1. I want to achieve a specific Goal, and as a Role I should be able to accomplish this by performing Functionality.

2. A Role invokes Feature to cause a Benefit

BDD Example with Groovy

Easyb is a behavior driven development framework for the Java platform. By using a specification based Domain Specific Language, easyb aims to enable executable, yet readable documentation.

Example Stories with easyb

Example Specification with easyb

BDD On Java using JBehave

JBehave is a framework for Behaviour-Driven Development Behaviour-driven development (BDD) is an evolution of test-driven development (TDD) and acceptance-test driven design, and is intended to make these practices more accessible and intuitive to newcomers and experts alike.

It shifts the vocabulary from being test-based to behaviour-based, and positions itself as a design philosophy.

and here is another example with java using JBehave.

Source:
BDD - http://behaviour-driven.org/
Easyb - http://easyb.org/
JBehave -http://jbehave.org/

Challenges of Scaling Agile in Enterprises

2010-03-08T12:07:00.000-08:00

When it comes to scaling agile to the enterprise, there are two classes of challenges that must be addressed. The first- the challenges inherent in agile itself – present limits of technology because of the fixed rule bases and assumptions built into the methods.

The second – those imposed by the enterprise – are impediments that likely exist within the enterprise that will otherwise prevent the successful application of the new methods. Both types of challenges must be successfully addressed in order for the enterprise to achieve the full benefits of agile.

Here is brief introduction some of the issues which should be consider for scaling agile in enterprise.

Small Team size
XP and Scrum recommend teams of 8 or fewer people including product owner/managers or other customer proxy, developers and testers. In many cases, teams decide that even that number is too large, and they may subdivide into components or feature team of three to five people. To an enterprise with 1000 practitioners to deploy, thinking in terms of managing literally hundreds of teams boggles the mind, and there arises the need to understand how to fit this new model into the existing organizational hierarchy.

Customer is Integral to Team
XP Success in particular in a number of IT like environments where the customer was literally down the hall the business analyst who worked with that department was willing and able to participate in the fine grained, detailed review that stories and customer written acceptance tests require.

for many enterprises however, this is not the case, the customer maybe be remote or may not have the skills or time available to participate in such a manner. Or perhaps the enterprise is a large ISV where the application has tens of thousand of users and where there is no single customer to satisfy. Every customer is different, and the larger ones speak loudly indeed, and the information flow is attenuated through many groups before it reaches the team.

Collocation
Much of the productivity of agile comes from the paring contexts, daily stand up, visual signaling of stories and status, and constant informal communication that characterizes these methods. Developers, product owners, and testers are together, not separated by time zones and language barriers.

At scale, collocation is impractical even for large teams in the same building and other mechanisms must be devised. Moreover it is likely that many team members are in different countries and different time zones and perhaps even speak different languages.

Architecture Emerges
Agile and more specifically, XP trades off the ability to refactor code quickly with the cost and investment necessary to create a more forward looking architecture, some architectural runway, that coordinate the efforts of distributed teams.

With the large scale systems however, the refactor cost curve may not be the case, because hundreds of person years may be necessary to get even a first release deployed. Since agile generally does not coach how to approach architecting larger scale systems, that apparent limitation is often a real impediment for agile adoption.

Lack of requirements analysis and documented Specifications
Agile practice of working on a few stories at a time is a wonderfully focusing mechanism for the team. But in larger system, what drive these stories into existence? Who says these stories are the right stories? Will the summation of all these stories actually meet our customer needs?

Culture and Physical Environment
Effective agile environments look and sound different. Most team members work in an open or nearly open environment and often their manager and supervisors have left their offices and joined the team at those tables. Also the process does not look very efficient because there is a certain informality to stories posted on walls, whiteboards shoved hither and you, and teams of people seeming to do far more talking than coding.

Existing Formalized Policies and Procedures
As organization grow, they tend to put in infrastructure to control and measure project, program, human resource and other assets, these include all of the procedure, policies that organization use. And this is part of organization culture and knowledge base which is gained in past project.

Management Style – Fixed Schedule, Fixed functionality Mandates
If a mandate comes to team to deliver functionality X in period of Y with resources Z, then by definition it does not meet agile fixed time, variable scope mantra. And the teams will be discouraged right out of the starting gate.

This mode of management is routine in our industry and most likely, this imprediment will have to be addressed head on. It’s true that the teams would like to be able to predict exactly when they can deliver what functionality, but they know they can not, and it seems inherently silly to them when management just says, “Make it be so”.

People Organized by Discipline rather than Product line
The word Organization in itself can be a root cause of impediments to agile, for the enterprise is surely organized in some fashion already, and most likely it is organized along functional (product management, architecture, developers) rather than product or business application lines. In agile, teams quickly reorganize to assure they have a full complement of the resources necessary to define/build/test and deliver a component of feature.

This require dedicated (not heavy multiplexed) resources for the project, or else the team will fail to meet its commitment to the iteration. Reorganization typically requires redefinition of what makes a team a team in the enterprise.

High Degree of Distribution
The enterprise is an enterprise because it has grown with its successes. For most enterprises, success often involves acquisitions of teams or product lines or existing IT organization along the way. Rarely are these teams collocated and the large the enterprise, the less likely they are to be together in one place.

Moreover, raw size alone prevents pure collocation because there is no way physically to collocation even 100 people in one workspace, so the problem of distributed teams is endemic to agile at scale.

Source: Scaling Software Agility Best Practices for Large Enterprises by Dean Leffingwell

J2EE 6 - Bean Validation

2010-03-06T10:41:00.000-08:00

Validating data is a common task that occurs throughout an application. For example, in the presentation layer of an application, you might want to validate that the number of characters a user enters in a text field is at most 20 characters or that the number a user enters in a numeric field is positive. If you set those constraints, you probably want the same data to be validated before it's used in the business logic of the application and when the data is stored in a database.

Developers often code the same validation logic in multiple layers of an application, something that's time consuming and error-prone. Or they put the validation logic in their data model, cluttering it with what is essentially metadata.

Bean Validation, JSR 303 makes validation simpler and reduces the duplication, errors, and clutter that characterizes the way validation is often handled in enterprise applications. Bean Validation affords a standard framework for validation, in which the same set of validations can be shared by all the layers of an application.

Specifically, Bean Validation offers a framework for validating Java classes written according to JavaBeans conventions. You use annotations to specify constraints on a JavaBean — you can annotate a JavaBean class, field, or property.

You can also extend or override these constraints through XML descriptors. A validator class then validates each constraint. You specify which validator class to use for a given type of constraint.

Here, for example, is part of a class that declares some constraints through Bean Validation annotations:

The @NotNull annotation specifies that the annotated element, addressline1, must not be null. The @Size annotation specifies that the annotated elements, addressline1 and addressline2, must not be longer than the specified maximum, 30 characters.

When an Address object is validated, the addressline1 value is passed to a validator class that is defined for the @NotNull constraint as well as to a validator class defined for the @Size constraint. The addressline2 value is also passed to the validator class for the @Size constraint. The pertinent validator classes perform the validations.

Both the @NotNull and @Size constraints are built into the Bean Validation framework so you do not need to define validator classes for them. However, you can add your own constraints to the built-in constraints, in which case, you need to define validator classes. For example, you can define a constraint named @ZipCode as follows:

Then you can specify the @ZipCode constraint on a class, field, or property just like any other constraint. Here is an example:

When an Address object is validated, the zipCode value is passed to the ZipcodeValidator class for validation. Notice that the constraint definition includes another constraint: @Size(min=5, max=5).

This means that an element annotated by the @ZipCode annotation must be exactly 5 characters in length. The element is validated against this constraint in addition to the primary constraint check that ZipcodeValidator performs.

Bean Validation allows you to create a constraint that is composed of other constraints. In fact, any composing constraint can itself be composed of constraints. Notice too that the constraint definition specifies an error message to be returned if the constraint fails the validation check. Here, the error message is "Wrong zipcode".

You can also use Bean Validation to validate an entire object graph in a straightforward way. An object graph is an object composed of other objects. If you specify the @Valid annotation on the root object of an object graph, it directs the pertinent validator to recursively validate the associated objects in the object graph. Consider the following example:

When an Order object is validated, the Address object and the associated objects in its object graph are validated too.

To meet the objective of sharing the same set of validations across all the layers of an application, Bean Validation is integrated across the Java EE 6 platform.

For example, presentation-layer technologies such as JSF and enterprise-layer technologies such as JPA have access to the constraint definitions and validators available through the Bean Validation framework. You no longer need to specify constraints in multiple places and in multiple ways across the layers of an application.

Source: java.sun.com - J2EE 6 Overview

J2EE 6 - Contexts and Dependency Injection

2010-03-06T10:21:00.000-08:00

Contexts and Dependency Injection for the Java EE Platform (CDI), JSR 299 is a technology that supplies a powerful set of services to Java EE components. These services allow Java EE components, including EJB session beans and JavaServer Faces (JSF) managed beans, to be bound to lifecycle contexts, to be injected, and to interact in a loosely coupled way by firing and observing events. Perhaps most significantly, CDI unifies and simplifies the EJB and JSF programming models. It allows enterprise beans to replace JSF managed beans in a JSF application.

In essence, CDI helps bridge what was a major gap between the web tier of the Java EE platform and the enterprise tier. The enterprise tier, through technologies such as EJB and JPA, has strong support for transactional resources.

For example, using EJB and JPA you can easily build an application that interacts with a database, commits or rolls back transactions on the data, and persists the data. The web tier, by comparison, is focused on presentation. Web tier technologies such as JSF and JavaServer Pages (JSP pages) render the user interface and display its content, but have no integrated facilities for handling transactional resources.

Through its services, CDI brings transactional support to the web tier. This can make it a lot easier to access transactional resources in web applications. For example, CDI makes it a lot easier to build a Java EE web application that accesses a database with persistence provided by JPA.

Let's look at some key parts of a web application that uses CDI services. The application, which processes user login and user logout requests, includes both JSF and EJB components. Here is the code for an input form on a JSF page that displays a login prompt for the web application:

As you can see from the code, the login prompt displays fields for a user to enter a user name and password. It also displays a Login button and a Logout button. Notice the unified expression language (EL) expressions such as #{credentials.username} and #{login.login}. These expressions refer to beans, named credentials and login.

Note that CDI builds on a new concept introduced in Java EE 6 called managed beans, which is designed to unify all of the various types of beans in Java EE 6. A managed bean is a Java class that is treated as a managed component by the Java EE container. Optionally, you can give it a name in the same namespace as that used by EJB components.

A managed bean can also rely on a small number of container-provided services, mostly related to lifecycle management and resource injection. Other Java EE technologies such as JSF, EJB, and CDI build on this basic definition of a managed bean by adding services. So for example, a JSF managed bean adds lifecycle scopes, an EJB session bean adds services such as support for transactions, and CDI adds services such as dependency injection. In CDI a managed bean or simply a bean is a Java EE component that can be injected into other components, associated with a context, or reached through EL expressions.

You declare a managed bean by annotating its class with the javax.annotation.ManagedBean annotation or by using one of several CDI annotations such as a scope annotation or a qualifier annotation. The annotation-based programming model makes it possible for a bean to begin as a POJO and later become another type of Java EE component such as an EJB component — perhaps to take advantage of more advanced functionality, such as transactional and security annotations or the instance pooling facility offered by EJB containers.

For example, you can turn a POJO into a stateful session bean by adding a @Stateful annotation to the object. Clients that use CDI to access a bean are unaffected by the bean's transition from POJO to EJB.

Any bean can be bound to a lifecycle context, can be injected, and can interact with other beans in a loosely coupled way by firing and observing events. In addition, a bean may be called directly from Java code, or as in this example, it may be invoked in a unified EL expression. This enables a JSF page to directly access a bean, even a bean that is implemented as an EJB component such as a session bean.

In this application, a bean named Credentials has a lifecycle that is bound to the JSF request. The Credentials bean is implemented as a JavaBean as follows:

To request CDI services, you annotate a Java EE component with CDI annotations. The @Model annotation is a CDI annotation that identifies the Credentials bean as a model object in an Model-View-Controller (MVC) architecture. The annotation, which is built into CDI, is a stereotype annotation. A stereotype annotation marks a class as fulfilling a specific role within the application.

The application also includes a Login bean whose lifecycle is bound to the HTTP session. The Login bean is implemented as an EJB stateful session bean, as follows:

The @Stateful annotation is an EJB annotation that specifies that this bean is an EJB stateful session bean. The @TransactionAttribute and @RolesAllowed annotations are also EJB annotations. They declare the EJB transaction demarcation and security attributes of the annotated methods.

The @SessionScoped annotation is a CDI annotation that specifies a scope for the bean. All beans have a scope that determines the lifecycle of its instances and the instances of the bean that are made visible to instances of other beans. This is an important feature because components such as EJB components do not have a well-defined scope.

In particular, EJB components are not aware of the request, session, and application contexts of web tier components such as JSF managed beans, and do not have access to the state associated with those contexts. In addition, the lifecycle of a stateful EJB component cannot be scoped to a web-tier context.

By contrast, scoped objects in CDI exist in a well-defined lifecycle context that is managed by the Java EE container. Scoped objects may be automatically created when needed and then automatically destroyed when the context in which they were created ends.

Significantly, the state of a scoped object is automatically shared by clients that execute in the same context. This means that clients such as other beans that execute in the same context as a scoped object see the same instance of the object. But clients in a different context see a different instance.

The @SessionScoped annotation specifies that the scope type for the Login bean is session scope. Objects that are not associated with any of the usual scopes, but instead exist for the exclusive benefit of an object that triggered their creation, are said to be dependents of their owner. The lifecycle of these dependent objects is tied to that of the owner. In particular, a dependent object is destroyed whenever the owner is destroyed.

Beans typically acquire references to other beans through dependency injection. The dependency injection mechanism is completely type safe. CDI uses the annotations specified in JSR 330: Dependency Injection for Java for dependency injection.

One of those annotations, @Inject, identifies a point at which a dependency on a Java class or interface can be injected. The container then provides the needed resource. In this example, the Login bean specifies two injection points. The first use of the @Inject annotation in the example injects a dependency on the Credentials bean.

In response, the container will inject the Credentials bean into any instance of Login created within this context. The second @Inject annotation injects a dependency on the JPA EntityManager. The container will inject the EntityManager to manage the persistence context.

The @Produces annotation identifies the getCurrentUser() method as a producer method. A producer method is called whenever another bean in the system needs an injected object of the specified type. In this case, the injected object is the currently logged-in user, which is injected by the qualifier annotation @LoggedIn.

A qualifier identifies a specific implementation of a Java class or interface to be injected. In order to use a qualifier annotation, you first need to define its type as a qualifier. You use the @Qualifier annotation, another JSR 330 annotation, to do that. For example:

Let's return to the login prompt discussed earlier. When a user responds to the prompt and clicks the Submit button, CDI technology goes into action.

The Java EE container automatically instantiates a contextual instance of the Credentials bean and the Login bean. An instance of a bean that is bound to a context is called a contextual instance. JSF assigns the user name and password the user entered to the Credentials bean contextual instance. Next, JSF calls the login() method in the Login bean contextual instance.

This instance continues to exist for and is available to other requests in the same HTTP session, and provides the User object that represents the current user to any other bean that requires it.

This example demonstrates only some of the features in this powerful technology. Another feature enables beans to produce or consume events. Yet another lets you define interceptors that bind additional function across all bean types, or define decorators that apply the additional function to a specific bean type.

Source: java.sun.com - J2EE 6 Overview

J2EE 6 - Java API for RESTful Web Services (JAX-RS)

2010-03-05T08:34:00.000-08:00

Java API for RESTful Web Services (JAX-RS), JSR 311 enables you to rapidly build lightweight web services that conform to the Representational State Transfer (REST) style of software architecture. An important concept in REST is the existence of resources, each of which can be referred to with a global identifier, that is, a URI. In particular, data and functionality are considered resources that can be identified and accessed through URIs. To manipulate these resources, components of the network, clients and servers, communicate through a standardized interface such as HTTP and a small, fixed set of verbs — GET, PUT, POST, and DELETE — and exchange representations of these resources.

RESTful web services are web services built according to the REST architectural style. Building web services with the RESTful approach has emerged as a popular alternative to using SOAP-based technologies thanks to REST's lightweight nature and the ability to transmit data directly over HTTP.

JAX-RS furnishes a standardized API for building RESTful web services in Java. The API contributes a set of annotations and associated classes and interfaces. Applying the annotations to POJOs enables you to expose web resources. This approach makes it simple to create RESTful web services in Java.

The specification for the initial release of the technology, JAX-RS 1.0, was finalized in October 2008 and a reference implementation named Jersey is also available. Java EE 6 includes the latest release of the technology, JAX-RS 1.1, which is a maintenance release that aligns JAX-RS with new features in Java EE 6. Let's take a look at a RESTful web service that uses JAX-RS.

In this example, the ItemsResource class is a web service that manages a set of items. The imports in the class are for JAX-RS 1.1 annotations, classes, and interfaces.

The @Path annotation specifies a relative path for the resource, in this case "items". The URI for the class resource is based on the application context. So if the application context for this example is http://example.com, the URI for the class resource is http://example.com/items.

This means that if a client directs a request to the URI http://example.com/items, the ItemsResource class will serve it. The @GET annotation specifies that the annotated method, here the listItems() method, handles HTTP GET requests. When a client directs an HTTP GET request to the URI for the ItemsResource resource, the JAX-RS runtime invokes the listItems() method to handle the GET request.

Notice the @Produces annotation. It specifies the MIME media types that the methods in the resource can produce and return to the client. In the ItemsResource example, the @Produces annotation specifies MediaType.APPLICATION_XML. The MediaType class is an abstraction of a MIME media type. Constants supplied to the class identify the particular media type to be abstracted. The MediaType.APPLICATION_XML specification is an abstraction of the MIME media type for XML content, "application/xml".

Annotations such as @Produces suggest some of the content type translation that JAX-RS handles automatically. For example, the listItems() method returns a Java object of type Items. JAX-RS automatically translates that Java type to the "application/xml" MIME type to be used in the HTTP response to the client. Note that the translation is automatic only if the returned type is supported by default.

For instance, if Items is a JAXB-annotated bean, then the translation would be automatic. However, if Items is a POJO, you would need to implement a MessageBodyReader to handle the serialization. You can also specify a @Produces annotation on a method. In that case, the MIME type you specify on the method overrides the MIME types in any @Produces annotation that you specify on the class. For example, you could specify a @Produces annotation for the listItems() method as follows:

JAX-RS would then translate the Items Java type to the "text/plain" MIME type, which represents plain text, and return content of that type in the HTTP response to the client.

The @POST annotation specifies that the annotated method, in this case, the create() method, responds to HTTP POST requests. In this example, the method creates a new item, perhaps in a database, and then returns a response indicating that it created the new item. When a client directs an = HTTP POST request to the URI for the ItemsResource resource, the JAX-RS runtime = invokes the create() method to handle the POST request.

Notice that the @Consumes annotation is specified on the create() method. The annotation specifies the MIME media types that the methods in the resource can accept from the client. As is the case for the @Produces annotation, if you specify @Consumes on a class, it applies to all the methods in the class. If you specify @Consumes on a method, it overrides the MIME type in any @Consumes annotation that you specify for the class.

In the example, the @Consumes annotation specifies that the create() method can accept XML content, that is, the MIME type "application/xml". Here the type translation is from MIME type to Java type. When a client submits XML content in a POST request to the URI for the ItemsResource class, JAX-RS invokes the create() method and automatically translates the incoming XML to the Item Java type required for the method's argument.

JAX-RS also includes a number of utility classes and interfaces that further simplify actions related to building and using RESTful web services in Java. You've already seen one of them: MediaType, a class for abstracting MIME media types. Some others are:

UriInfo, an interface for accessing URI information. In this example, the @Context annotation injects the UriInfo interface into the uriInfo field of the ItemsResource class.
UriBuilder, a class for building URIs from their components
Response, a class represents an HTTP response
Response.ResponseBuilder, a class that builds Response objects, in accordance with the well-known Builder Pattern

These classes and interfaces are used in the following statements in the example:

The first statement builds a URI for the new item. The getRequestUriBuilder() method is a UriInfo method that creates a UriBuilder object. The path() and build() methods are UriBuilder methods that together construct the URI for the new item.

The second statement creates a Response object for the new item to be returned to the client. The created method is a Response method that creates a Response.ResponseBuilder object. The build() method is a Response.ResponseBuilder method that creates the Response object for the new item. This object delivers metadata to the JAX-RS runtime to construct the HTTP response.

These utility classes and interfaces hide a lot of the complexity of HTTP programming — another reason why using JAX-RS is a simple way to build RESTful web services. However, this simplicity also extends beyond web services. JAX-RS can simplify the development of many types of HTTP-aware web applications. For example, if you need to build a web application that examines HTTP headers, you can probably code it in a much simpler way by using JAX-RS rather than other approaches.

JAX-RS has other convenient features. For example, JAX-RS includes a number of parameter-based annotations to extract information from a request. One of these is @QueryParam, with which you can extract query parameters from the Query component of a request URL. Some other parameter-based annotations are @MatrixParam, which extracts information from URL path segments,@HeaderParam, which extracts information from HTTP headers, and @CookieParam which extracts information from the cookies declared in cookie related HTTP headers.

Source: java.sun.com - J2EE 6 Overview

Introduction to Database Marketing

2010-03-05T06:49:00.000-08:00

The purpose of marketing is to enable the firm to enhance customer value. In today’s competitive, information-intensive, ROI-oriented business environment, database marketing has emerged as an invaluable approach for achieving this purpose. The applications of database marketing are numerous and growing exponentially. Here are a few examples:

• “Internet Portal, Inc.” determines which of its customers will be most receptive to targeted efforts to increase their usage of the portal. Perhaps more importantly, it determines which customers will not be receptive to these efforts.

• “XYZ Bank” decides which of its many financial products should be marketed to which of its current customers.

• “ABC Wireless” develops the ability to predict which customers are most likely to leave when their contract runs out, and designs a “churn management program” to encourage them to stay.

• UK Retailer Tesco develops thousands of customized promotion packages it mails to its 14 million customers (Rohwedder 2006).

• Best Buy has identified the major segments of customers who visit its stores. It then (1) tailors its store in a particular locality to fit the representation of the segments in that locality, and (2) trains its store personnel to recognize which segment a particular customer belongs to, so the customer can be serviced appropriately (Boyle 2006).

• Catalogers routinely use “predictive models” to decide which customers should receive which catalogs.

• “E-tailer Z” uses “recommendation engines” to customize which products it “cross-sells” to which customers.

• Dell Computer uses data analyses of prospects to improve its customer acquisition rate (Direct Marketing Association 2006).

These are but a few examples of database marketing in action. The common theme is that all of them are based on analyzing customer data and implementing the results.

Database marketing is the use of customer databases to enhance marketing productivity through more effective acquisition, retention, and development of customers.

Each phrase in this definition is carefully chosen. First, database marketing is fundamentally about using of customer databases. The “customer” can be either current customers or potential customers. Firms have data on their current customers’ purchase behavior and demographic and psychographic information, as well as the firm’s previous marketing efforts extended to these customers and their response to them.

For potential customers prospects firms may be able to obtain data on customer demographics and psychographics, as well as purchase history data, although obviously not in the same depth as available for their current customers.

Second, database marketing is about marketing productivity. In today’s results-oriented businesses, senior management often asks the simple question, “Do our marketing efforts pay off?” Database marketing attempts to quantify that effectiveness and improve it. It does this through effective targeting.

The retail pioneer John Wannamaker is credited with saying, “I know half of my advertising doesn’t work; I just don’t know which half.” Thinking more broadly, in terms of marketing rather than advertising, database marketing identifies which half of the firm’s marketing efforts is wasted. It does this by learning which customers respond to marketing and which ones do not. The responsive customers are the ones who are then targeted.

Third, database marketing is about managing customers. Customers must be acquired, retained, and developed. Acquiring customers means getting an individual who currently does not do business with the company to start doing business with the company. Retention means ensuring the current customer keeps doing business with the company. Development means enhancing the volume of business the retained customer does with the company.

For now, the important point is to recognize that database marketing is concerned with all three elements of customer equity. The Dell example above involves customer acquisition. The ABC Telecom example involves customer retention. The XYZ Bank, Tesco, and E-tailer Z examples involve customer development.

Why Is Database Marketing BecomingMore Important?

The indications are that the database marketing industry is huge and increasing. The question is, why? We hypothesize five major classes of reasons:

• Information technology: Companies now have the ability to store and manipulate terabytes of data. While the software to do so is expensive, the capabilities are dramatic.

• Growth of the Internet: The Internet is a data-collection “machine.” Many companies that previously could not collect and organize data on their customers can now do so through the Internet.

• Lower productivity of mass marketing: While there are no good statistics on this, there is the belief that mass advertising and non-customized marketing efforts are eliciting poorer response, while costs are increasing and margins are declining.

One can write the profitability of a marketing campaign as Π = Npm−Nc, where N is the number of customers reached by the campaign, p is the percentage that respond, m is the contribution margin when they respond, and c is the cost of contact per customer. For a campaign to be profitability, we need p > c/m. Unfortunately, all three of these terms are moving in the wrong direction. Response is lower (p), costs are higher (c), and margins are lower (m). Database marketing targets customers for whom response is maximal, helping the profit equation to remain in the black.

• Marketing accountability: Results-oriented senior managers are requiring all business functions to justify their existence, including marketing. No longer is it taken on faith that “marketing works” or “marketing is a cost of doing business.” The demands of senior managers for proven results feed directly into database marketing’s emphasis on analyzing data and measuring results.

• Increasing interest in customer relationships: Companies are more concerned than ever about their relationship with the customer. They see their products commoditizing and customer loyalty wilting away. Database marketing is a systematic way to improve customer relationships.

• Establishing a competitive advantage: Companies are always trying to determine what will be their source of competitive advantage. Perhaps that source lies in the data they have on their own customers, which allows them to service those customers better through database marketing.

Source: Database Marketing Analyzing and Managing Customers - Springer

Scrum - The Rugby Strategy in Product Development

2010-01-15T02:01:00.000-08:00

Scrum is not primarily about software development. It is a method for managing product development that can be wrapped around any specific technology, including software. Scrum as it exists today grew from its beginnings in Japan in the mid-1980s. The name “Scrum” is from the game of rugby and refers to a strategy used to get a ball back into play. The Scrum process is incremental, just as with other Agile methods.

Scrum practices

The Scrum Master
“The Scrum Master is responsible for the success of Scrum” . Although Scrum defines this as a new role, in traditional projects its responsibilities are often taken on by an existing position such as project manager or team leader. The primary responsibilities of the Scrum Master are to:

◗ Ensure that the Scrum practices are followed and that the values behind Scrum drive enactment of the process.
◗ Work with management and the customer to identify the individual who will take on the role of “Product Owner.”
◗ Facilitate each of the other practices
◗ Be the interface point among management, the customer, and the Scrum team. Of primary importance are:
◗ Communicating project status;
◗ Removing impediments to progress

Product Backlog
“Product Backlog is an evolving, prioritized queue of business and technical functionality that needs to be developed into a system”. The Product Backlog is the sum total of the work that remains to be done on the project and includes everything from major features to bug fixes. Any stakeholder in the project can contribute to the Product Backlog at any time, but it is the Product Owner who has the primary responsibility for determining the priority of backlog items.

The primary measure of progress in a Scrum project is the change in the number of items in the Product Backlog over time. It may grow in early Sprints as stakeholders gain an understanding of the system being built, but ultimately, a pattern of steady decrease in the size of the Product Backlog is expected. If this does not materialize, or if it is not fast enough, then hard decisions must be made about the project’s scope.

Scrum Teams
“A team commits to achieving a Sprint goal. The team is accorded full authority to do whatever it decides is necessary to achieve the goal” . Almost all software development involves teams. The key difference with Scrum is that the team freely commits to what they believe they can produce during each Sprint, and they are empowered to make whatever decisions they must to fulfill those commitments. This level of autonomy is foreign to most organizations.

Daily Scrum Meetings
“Software development is a complex process that requires lots of communications.
The Daily Scrum meeting is where the team comes to communicate”. The Daily Scrum (the defining feature of Scrum) is a short 15-minute meeting that takes place every working day. It is the forum where team members exchange information and others may come to listen but not speak. To keep the meeting short, all deliberation and discussion is relegated to meetings of interested people after the Daily Scrum. During the Daily Scrum, each team member answers three questions:

◗ What have you done since the last Scrum?
◗ What will you do between now and the next Scrum?
◗ What got in your way of doing work?

The third question provides the Scrum Master with the information he or she needs to be effective in removing impediments to progress and ensuring the team continues to be productive.

Sprint Planning Meeting
“Customers, users, management, the Product Owner, and the Scrum Team determine the next Sprint goal and functionality at the Sprint Planning meeting. The team then devises the individual tasks that must be performed to build the product increment”.

Each 30-day Sprint begins with this planning meeting. The critical outputs of this meeting are:

◗ Sprint Goal—The objective that is to be achieved during this Sprint.
◗ Sprint Backlog—The subset of the Product Backlog that will be completed
during the Sprint.

The Product Owner is the sole arbiter of the priority of the Product Backlog items. But only the Scrum Team can commit themselves to completing specific work. The Sprint Planning meeting is the forum where lobbying and negotiation take place. At its conclusion, all stakeholders will have agreed to a Sprint Goal and Sprint Backlog to which the Team is willing to commit.

Sprint
“The team works for a fixed period of time called a Sprint”. After the negotiations of the Sprint Planning meeting, the Scrum Team has full authority to complete the 30-day Sprint by doing whatever they feel is necessary. During the Sprint, the team self-organizes and self-directs, and their authority even extends to being able to:

◗ Change the functionality to be delivered by the Sprint as long as the Sprint Goal is still achieved.
◗ Abort the Sprint if new information leads them to believe its Goal or Backlog is no longer achievable or relevant.

Assuming the team does not abort the Sprint, it ends with the delivery of the promised executable product increment.

Sprint Review
“The Sprint Review meeting is a four-hour informational meeting. During this meeting, the team presents to management, customers, users, and the Product Owner the product increment that it has built during the Sprint”. This meeting provides a concrete picture of the progress achieved during the Sprint and lays the foundation for the next Sprint Planning meeting.

References
- Takeuchi, H., and I. Nonaka, “The New New Product Development Game,” Harvard Business Review, January 1986.
- Schwaber, K., and M. Beedle, Agile Software Development with Scrum, Upper Saddle River, NJ: Prentice-Hall, 2002.

22 Lean Software Development tools

2010-01-15T01:43:00.000-08:00

LD is characterized by seven lean principles that are elaborated into 22 Lean Software Development tools.

Eliminate Waste
Tool 1: Seeing Waste—You should looking for waste in these parts of your development process:
◗ Partially done work (e.g., unimplemented designs);
◗ Extra processes (steps in your process that do not add value);
◗ Extra features (things for which the customer did not ask);
◗ Task switching (people assigned to multiple projects);
◗ Waiting (for hand-offs);
◗ Motion (hand-offs);
◗ Defects.

Tool 2: Value Stream Mapping—Identify all the steps in your process that add value to the product, and for each of those steps, identify how much time is required to add the value and how much time is wasted in waiting. Note that some steps that do not directly add value are necessary (e.g., Configuration Management mitigates risks).

Amplify Learning
Tool 3: Feedback—Design your process so that participants receive feedback as often as possible. For example:
◗ Test early and often;
◗ Show users the evolving system;
◗ Prototype instead of analyzing.

Tool 4: Iterations—Iterative time-boxed development is the best way to converge on the ultimate solution. Often this results in negotiation of project scope or duration. But such negotiation is good if done early in the project, when adjustments are less painful.

Tool 5: Synchronization—Every development method results in synchronization problems (e.g., collective code ownership can result in multiple people changing the same program at the same time.) The authors recommend several synchronization mechanisms for software projects.

Tool 6: Set-Based Development—When faced with a difficult problem, it is often good to think about the solution space instead of specific solutions. The authors recommend these steps:
◗ Develop multiple options (e.g., brainstorming ideas);
◗ Communicate constraints;
◗ Let the solution emerge.

Decide as Late as Possible
Tool 7: Options Thinking—In financial markets, you can purchase options that, for a price, allow you to delay making a decision until a later date. The same model can be used in Agile software development. By incurring some added cost, options can be held open, allowing decisions to be delayed until more information is available.

Tool 8: The Last Responsible Moment—Options Thinking (Tool 7) delays commitment to a decision until the last responsible moment. The authors define that as “the moment at which failing to make a decision eliminates an important alternative” .

Tool 9: Making Decisions—Good decisions are more likely to be made when the options can be narrowed by the application of a set of rules. The authors recommend that decisions in software development always be made in light of the seven lean principles

Deliver as Fast as Possible
Tool 10: Pull Systems—The best way to ensure that each team member knows exactly what he or she should do each day is to ensure that the environment provides automatic prompting. For example:

◗ Short iterations with well-defined deliverables provide near-term targets for people to work toward.
◗ Daily stand-up meetings keep everyone aware of progress toward that target.
◗ Information radiators (publicly posted charts) keep everyone aware of the big picture.

Tool 11: Queuing Theory—Much waste can be attributed to people waiting for constrained resources. The authors suggest using queuing theory to analyze all bottlenecks in your software process and determine ways to minimize the cycle time.

Tool 12: Cost of Delay—In many software projects, the cost of delaying delivery (even when it is significant) is often invisible to the project team and sometimes to managers. The authors recommend developing a financial model for the project that will allow all stakeholders to make appropriate trade-off decisions, especially when cost and delivery date are in conflict.

Empower the Team
Tool 13: Self-Determination—The people who do the work are in the best position to improve their processes and eliminate waste. Although Quality Circles have fallen out of favor, this philosophy still has value in guiding process improvement efforts.

Tool 14: Motivation—Motivating a project team to perform well has many dimensions. The authors stress that the team must have:

◗ Purpose—A shared goal that they all believe in.
◗ Belonging—They feel that each person is a part of the team.
◗ Safety—Mistakes are corrected without punishing people.
◗ Competence—The team feels they able to complete their tasks.
◗ Progress—Each person can see regular progress on the project.

Tool 15: Leadership—Project management and technical leadership are two distinct skill sets. A successful team needs both, and it is rare for those two attributes to exist in the same person.

Tool 16: Expertise—Every software project requires a variety of expertise (e.g., domain, user interface, database, project management, writing). This tool is about identifying communities of expertise and ensuring that they are available to the project.

Build Integrity In
Tool 17: Perceived Integrity—Perceived Integrity is the system’s integrity from customers’ viewpoint. Do they perceive it to be useful and well designed? With appropriate interaction between the team and customer, the team can be sure they build the right system.

Tool 18: Conceptual Integrity—Conceptual Integrity is an important component of Perceived Integrity (Tool 17). Building the system right (as differentiated from building the right system) involves such concepts as architecture, consistency, and elegance.

Tool 19: Refactoring—Programmers should improve the software any time they see the opportunity to do so. Refactoring refers to redesign of the system to improve the program’s Perceived and Conceptual Integrity (Tools 17 and 18).
Also, after many changes have been made to software, it tends to become brittle. (That is, over time, it becomes more and more difficult to make any changes to the software without breaking it in unforeseen ways.) Brittle code is a Conceptual Integrity problem that Refactoring remedies.

Tool 20: Testing—Testing provides an important feedback mechanism. (See Tool 3.) Both validation (ensuring the right system is built) and verification (ensuring the system is built right) must be done throughout the development process, not just at the end. That way, the team gets the feedback they need to ensure their system’s integrity, both Perceived (Tool 17) and Conceptual (Tool 18).

See the Whole
Tool 21: Measurements—The authors focus on the various problems associated with measurement (e.g., suboptimization) and their sources. In essence, this tool is about focusing measurement where it is effective: at higher-level aggregations of data. This focus has the dual benefit of protecting individual engineers from management’s abuse of their personal data and focusing attention on data that is useful to the organization as a whole.

Tool 22: Contracts—Contracts are a fact any time the supplier and customer are different companies. Because these relationships can be so complex and varied, the authors discuss a variety of contractual arrangements and the positive and negative effects of each.

References
- Poppendieck, M., and T. Poppendieck, Lean Software Development: An Agile Toolkit, Reading, MA: Addison-Wesley, Inc., 2003.
- Cockburn, A., Agile Software Development, Reading, MA: Addison- Wesley, Inc., 2002.

Agile Simplified with Feature Driven Development

2010-01-15T00:51:00.000-08:00

FDD differs from other Agile methods in its focus on upfront design and planning. As can be seen from the FDD process diagram in this picture, the Object Model, feature list, and planning are done once at the beginning of the project, and iterations are essentially an incremental building of identified features. This is not to say that the model, feature list, and plans never change; rather, it indicates that evolution of those items is not an inherent part of this method.

FDD practices

Domain Object Modeling

FDD begins with the construction of a relatively detailed object model for the system to be built. This model is not intended to provide all the design details for the features. Rather, its intention is to force all stakeholders’ assumptions out into the open and provide a road map for the project. Although the initial Object Model is built at the beginning of the project, each increment of development results in updates to it as details are filled in and corrections are made. Thus, the Object Model is a continually evolving picture of the product, as it is currently understood.

Developing by feature

The Object Model identifies all the expected classes in the system, but development is not done class by class. Rather FDD requires that development be done a feature at a time.

FDD defines a feature this way. “A feature is a small, client-valued function expressed in the form: Action - Result - Object

with the appropriate prepositions between the action, result, or object”. An example of a feature might be, “Retrieve the medical records of a patient.” As can be seen from the example, a feature is small and can normally be developed in a few hours or days. FDD defines the upper limit on feature size at 2 weeks. Because developing a feature can require additions or changes to several classes, the next two practices are included in FDD to define how this activity is managed.

Class (code) ownership FDD prescribes that a single developer owns each class. This is diametrically opposed to XP’s practice of “collective ownership” and grows from a different philosophical basis. FDD depends on the class owner to have a full and detailed understanding of all that his or her class does and contains. The philosophy is that such an individual will be more efficient at making changes to a class than anyone else on the project team. Of course, the danger in this practice is that the loss of a team member can be catastrophic. This class ownership practice essentially requires that the class owner be directly involved in the development of each feature that affects his or her class. The next practice (feature teams) defines how this requirement is managed.

Feature teams Each feature is implemented by a team of project members, consisting of the feature owner (a Chief Programmer) and the owners of all the classes affected by the feature. With direct participation of the expert on each class involved in the feature, that feature should be implemented not only very quickly but also in the most effective way. If, during a feature implementation, the team determines that another class must change, the owner of that class is simply added to the feature team. Thus, feature teams are a dynamic part of FDD, forming and disbanding on a daily or weekly basis as the development of each feature is undertaken and completed. Multiple feature teams will likely be operating at any one point in time, and each individual may be operating on more than one feature team at a time. A feature team has not completed its work until it has verified its feature and integrated it into the current product baseline.

Inspections The primary purpose of an inspection is to detect defects in designs and code, but FDD identifies two other critical results of inspections. First, it mitigates the risks involved in the class ownership practice by ensuring that many team members have a good understanding of each class. And second, it is a forcing function to ensure that the project’s coding standards are adhered to consistently.

Regular build schedule FDD does not prescribe any particular timing for builds, only that they be “regular.” How often builds are done must be defined for each project based on the project’s needs and constraints and the environment. Even so, FDD envisions that builds are done no less often than weekly, and possibly daily or continuously. Regular builds are done to maintain an up-to-date system comprised of all the features that have been developed to date. This current system then can be used as the baseline for testing each new feature, as a basis for writers to develop documentation, and as a continually available demonstration for customers and project sponsors.

Configuration Management This FDD practice acknowledges the importance of good CM to the success of an FDD project. The dynamism of the feature teams and regular build schedule make it critical that the project carefully manage its artifacts. But FDD does not prescribe any specifics about CM. It simply directs that the project team practice a level of CM rigor that is appropriate to the project size, complexity, and scope.

Reporting/Visibility of results FDD uses a unique mechanism for tracking and reporting the project’s status so that the common “90% complete” syndrome is avoided. This mechanism uses the project’s feature list and feature development milestones along with some weighting factors for those milestones.

FDD defines the milestones for each feature to be:

◗ Domain walkthrough—complete;

◗ Design—ready for inspection;

◗ Design inspection—and any rework and reinspection complete;

◗ Code—ready for inspection;

◗ Code inspection—and any rework and reinspection complete;

◗ Promote to build—all feature code checked in and ready for the next build.

Each FDD project team uses its historical data to assign a weight to each milestone. For example, if their data shows that they average 4% of their time in design walkthroughs, then that milestone is given a weight of 0.04 for every feature in the project.

References

Palmer, S. R., and J. M. Felsing, A Practical Guide to Feature-Driven Development, Upper Saddle River, NJ: Prentice-Hall, 2002.

The 12 Practices of Extreme Programming

2010-01-14T08:35:00.000-08:00

The Planning Game
In XP, planning (characterized as “The Planning Game”) is both iterative and collaborative. It is iterative in that the “game is played” at the beginning of each increment of development. It is collaborative in that it is played among the technical team members and the customer and management (referred to as
the business people). During the Planning Game for each increment, the business people and the technical people negotiate to establish an achievable plan that best meets the business needs. During this negotiation, the customer identifies what system features would be most valuable to develop next, the technical team determines the feasibility and effort for developing that functionality, and management ensures that the project stays within any relevant parameters (e.g., cost).

Small releases
An XP project team develops the system in the smallest reasonable chunks that provide demonstrable value for the customer. Because each increment should take only a few weeks to develop, the functionality that can be developed in that time is very constrained. But even when it is not feasible to actually release an increment for use, that increment is still demonstrated for customers so they can verify that it is progressing toward meeting their needs and appears to be usable.

Metaphor
An XP project’s Metaphor is the overall concept of the system that the team will build. The Metaphor uses similes to describe what the system will be like, in terms that are readily recognizable to both the development team and customer. This very high-level vision is the XP project’s overall requirement, and there is an expectation that it will remain relatively stable throughout the project.

The actual descriptions of the system features are documented separately in “Stories.” Each feature is documented in a separate Story that describes the feature’s essential attributes and actions. These descriptions are also quite brief, comprising only the text that can be written on a 3x5-inch card. The stories on an XP project tend to change dynamically as the development team and customer learn about the product they are building and gain a better understanding of the customer’s needs. The Stories, taken together with the project’s Metaphor, form the requirements from which the system is planned and developed.

Simple design
This practice is the heart of XP’s value of “Simplicity.” It advises that programmers avoid designing for the features that they “know” are coming (even if they will be implemented within the current increment). Instead, they must use the simplest possible design that will allow the current Story to be implemented.

XP is based on the philosophy that this “simple design” practice will result in less rework than designing for the future. This is because we are often wrong when we design for the future, and when we are, the rework can be significant. Beck sums up this practice: “If you believe that the future is uncertain, and you believe that you can cheaply change your mind, then putting in functionality on speculation is crazy”.

Test First
One of XP’s defining philosophies is “Test First.” This philosophy states that before a pair of programmers writes a single line of code, they must implement the automated tests that will be required to verify the Story they are about to write.

In addition, customers also develop a set of functional tests One of XP’s defining philosophies is “Test First.” This philosophy states that before a pair of programmers writes a single line of code, they must implement the automated tests that will be required to verify the Story they are to verify the Story according to their own needs. Coding the XP way consists of code a little, test a little, code a little, test a little—with the test results being the programmers’ measure of progress on the Story at hand. Work on the Story is not complete until all the tests run 100% clean. Successful integration is also defined in terms of these automated tests.

Refactoring
XP’s Refactoring practice prompts each pair of programmers to ask themselves before they begin work on a Story if there is a way to redesign the system so that the final result “is the simplest thing that could possibly work”. Then, after they have completed work on the Story, Refactoring prompts them to ask the same question again. In either case, if the answer is, “Yes,” then the redesign is implemented as part of the work on that Story. Refactoring is used to ensure that the “simple” designs that programmers start with do not grow into needless complexity as more Stories are added.

Pair Programming
The most visible practice of XP is Pair Programming. This practice calls for all technical work (from design to coding to test) to be done by a pair of programmers working together at a single workstation. Pairs form and change dynamically throughout the project according to the needs of each story. The member of each pair who is not typing has a very special job. That person is to be thinking about the wider impacts of what is being implemented— to be continuously asking questions like, “Is there a simpler way to do this?” “Do we need tests that we did not yet create?” “Are there defects in this code?” “Will this strategy work?” Pair Programming has been called the ultimate in peer reviews, because it entails a continuous, real-time review of everything that is done. This is one more example of XP’s strong emphasis on technical excellence.

Collective ownership
XP takes a position on code ownership that is quite different from the standard model of assigning responsibility for each code module to a specific person. XP goes to the opposite extreme—stating that code should never be “owned” by any individual. When anyone identifies a need to change any code, it is that pair’s responsibility to implement that change. Every member of the team owns all code collectively.

Continuous integration
XP goes beyond the concept of daily builds to require that integration of the product being built goes on continuously. As a pair completes each Story, their last step is to integrate their new and changed code (along with their automated tests) into the baseline system. They then run the entire test suite all of their own automated tests, as well as those for all the Stories that are already part of the system. If any test fails, the new Story and tests are backed out, and the pair must resolve the problems before they can try again. When all the tests run 100% clean, the Story has been successfully integrated and becomes part of the project baseline. Because there are several pairs of programmers working on different Stories at all times, this practice results in someone integrating something almost all the time.

40-hour week
XP calls for overtime to be rare. It explicitly suggests that overtime should not be worked 2 weeks in a row. The intention with this practice is to keep everyone fresh so they can continue indefinitely on an aggressive but sustainable pace.

On-site customer
One of the key members of an XP project team is the on-site customer. This is a person who will be a real user of the system being built, or some other person who can authoritatively stand in for the customer/user. This practice ensures that an appropriately knowledgeable person is continuously available to the team to review work, try things out, answer questions, and make implementation decisions when they are needed.

Coding standards
Several other XP practices (most notably Pair Programming and collective ownership) give rise to the importance of good coding standards. In a team that is collaborating this closely, appropriate standards are the only way to maintain order.Additionally this practice is support knowledge flow and knowledge management in your team.

References
Beck, K., Extreme Programming Explained, Reading, MA: Addison-Wesley Longman, Inc., 2000.

Nine Principles of Dynamic Systems Development Method

2010-01-14T08:12:00.000-08:00

Principle 1: Active user involvement is imperative.
DSDM’s strong focus on the business purpose of the system being developed requires that the ultimate users of the system be involved throughout the development project. This is because the system attributes that will make it fit for its purpose cannot be understood well enough in the project’s early stages to commit them to a detailed specification. Therefore, the only way to make appropriate detailed decisions and know that the evolving system is converging on the ideal of “fitness” is to fully involve the users throughout the project.

Principle 2: DSDM teams must be empowered to make decisions.
This principle does not give the team free reign to do whatever they wish. Rather, it advocates that the team be delegated the authority to make most of the day-to-day decisions as the project progresses. With active user involvement, such delegation can result in the team being able to move quickly and steadily toward system delivery. However, when a decision that must be made falls outside the team’s authority (e.g., cost overruns), DSDM recognizes the importance of raising such decisions to the appropriate authority.

Principle 3: The focus is on frequent delivery of products.
This principle means that the project’s progress should be measured by the production of tangible products, rather than by mere activity. The phrase “delivery of products” does not refer only to the incremental delivery of a working system to an end user. Products in this sense include any sort of work product that may be produced as the project moves forward (e.g., a specification, a throwaway prototype, a design document); and delivery could be simply within the project team. DSDM requires that as the project moves forward, it must produce artifacts that prove that progress is being made.

Principle 4: Fitness for business purpose is the essential criterion for acceptance of deliverables.
This principle is the practical manifestation of DSDM’s belief that specifying detailed requirements upfront is not helpful. By placing fitness for purpose above satisfaction of requirements, and by involving users consistently, DSDM zeroes in on the end user as the only one who can say whether or not the system as it is evolving is acceptable.

Principle 5: Iterative and incremental development is necessary to converge on an accurate business solution.
In an environment where it is assumed that the project’s end result cannot be foreseen in great detail, incremental development is the best insurance against the project going terribly awry. Incremental development is essentially an exercise in trial and error, where each new increment is presented to the user who validates (or invalidates) the direction the team has taken. “Converge” is the key word in the principle. It is assumed that the most direct path to the end product is not likely to be known, so DSDM engages in constant checking and correcting of the path to bring the project to a satisfactory end as quickly as is reasonably possible.

Principle 6: All changes during development are reversible.
If we agree that the project is practicing trial and error, then we must expect that there will indeed be errors from time to time. This principle gives us permission to discard erroneous work when necessary. Surely, we will try to salvage the good from a mistake, but we must recognize that there will be times when the most efficient path is to discard some work and try again.

Principle 7: Requirements are baselined at a high level.
The first three words of this principle, “Requirements are baselined,” represent a departure of DSDM from some other Agile methods. DSDM recognizes the importance to the project of stability in scope and direction. By baselining (freezing) the requirements at some level, stakeholders are establishing a stable basis for the team’s work. This does not mean that this baseline will not change; rather, it requires that serious deliberation precede any such change so that all stakeholders understand and agree to what would become the new requirements baseline. The last four words, “at a high level,” is the part of this principle that makes it agile. It leaves the details of what the requirements mean to be worked out between the team and user.

Principle 8: Testing is integrated throughout the life cycle.
Testing does not show up as a step in the DSDM life cycle because, like other Agile methods, DSDM promotes a strong quality-consciousness by all team members. Every task should include an appropriate verification or validation step like a review or test by a team member or user. This principle works together with principles 1, 4, and 5 to continually check the project’s progress toward its goal of a system fit for its business purpose.

Principle 9: A collaborative and cooperative approach between all stakeholders is essential.
This last principle is little more than the sum of the first eight. The only way that principles 1–8 can be applied successfully on a project is if all stakeholders accept DSDM and their roles as DSDM defines them. If any stakeholder does not agree (especially an influential stakeholder), then DSDM cannot work in that environment.

Reference
Stapleton, J., DSDM: Business Focused Development, London, England: Pearson Education, 2003.

Introduction to Adaptive Software Development

2010-01-14T07:55:00.000-08:00

Jim Highsmith’s ASD arose out of his understanding of Complex Adaptive Systems theory. He views a software project team as a complex adaptive system that consists of agents (team members and other stakeholders), environments (organizational, technological, process), and emergent outcomes (the product being developed).

The Adaptive Life Cycle

ASD’s Adaptive Life Cycle is composed of five steps. The initial step of “project initiation” is done once at the beginning of the project, and the final step of “final Q/A and release” is done once at the end.

The other three steps (adaptive cycle planning, concurrent component engineering and Quality Review) form the “Learning Loop” or “Adaptive Cycles” that are the heart of ASD. These Adaptive Cycles are described as:

◗ Mission-driven—each cycle must make progress toward the project mission;
◗ Component-based—more centrally focused on building things than on performing tasks;
◗ Iterative—going through the learning loop repeatedly is the key to progress;

◗ Time-boxed—the entire project as well as each cycle of the project is completed within a prescribed time period (with the developed functionality expanding or contracting to fit);
◗ Risk-driven—focusing on the highest-risk items first;
◗ Change-tolerant—designed to accommodate changes in each cycle.

Speculate: Project initiation
The project’s first step in speculation is an initiation workshop, from a few days to a few weeks in length, depending on the project’s size and scope. During initiation, the entire project team and the project sponsor or customer determine the project’s guiding parameters, including the project mission, objectives and constraints, the organization for the project, the system requirements, initial estimates of product size and scope, and key risks to the project.

Learning Loop
The initiation practice lays the groundwork for the project’s “Learning Loop.” This loop sees the project continually cycling from speculation to collaboration, to learning as the product emerges incrementally from the adaptive cycles (an iterative approach).

Speculate: Adaptive Cycle Planning
The first cycle of the project begins with the speculative practice we normally call “planning.” This includes these steps:

◗ Determine the project time-box.
◗ Determine the optimal number of cycles and the time-box for each. (Each cycle is usually established to be from a couple of weeks to a couple of months in length.)
◗ Write an objective statement for each cycle.
◗ Assign primary components to cycles.
◗ Assign technology and support components to cycles.
◗ Develop a project task list.

In each successive project cycle, Adaptive Cycle Planning consists of revisiting and revising these things as necessary, based on progress to date and what has been learned in earlier cycles.

Collaborate: Concurrent component engineering
This is where the real work gets done. The content of this phase of each cycle is planned in the Adaptive Cycle Planning phase and done to the extent possible within the planned time-box. Trade-offs may have to be made as the end of the time-box constrains what can be accomplished. This phase is shown as multiple boxes because team members or subteams are generally working concurrently and integrating their work products.

Learn: Quality Review
The end of each cycle is marked by learning activities, where the project team gains insight into progress to date and collects the information they need to perform any replanning at the start of the next cycle. Highsmith specifically recommends three learning activities:

Customer Focus Group Reviews—Because the objective of an ASD project is to converge on a system that meets its business purpose, evaluating the results of each cycle by the ultimate users is a critical learning activity. Highsmith recommends using a joint application development (JAD)-style facilitated workshop to collect input from the user community at the end of each cycle. During this workshop, developers would demonstrate and explain what has been built so far. Then users would try using the software and give developers their reactions to it.

Software Inspections—The primary objective of these inspections is to detect defects in the work products of the cycle, but they have a secondary benefit of ensuring that each team member is familiar with all the code that has been written.

Postmortems—The final step in each cycle is the postmortem, where team members evaluate the effectiveness of the processes they have been using as well as the project’s performance against its plan. If they identify problems, they also brainstorm ideas for solving those problems. The results of the postmortem are fed back (via the Learning Loop) to the planning phase for the next cycle.

Learn: Final Q/A and release
This phase is the final hand-off to the customer. Its focus is to put both the product and all pertinent information about it into the customer’s hands before disbanding the project team.

References
Highsmith, J. A., III, Adaptive Software Development, A Collaborative Approach to Managing Complex Systems, New York: Dorset House Publishing, 2000.

Motivation and Self Organization teams from Agile Perspective

2010-01-10T04:13:00.000-08:00

Motivated individuals

"Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done."

This principle highlights the Agile methods’ reliance on the motivation of each team member to achieve project success. Most of the practices of these methods make the assumption that team members are going far beyond simply following orders and doing what was assigned.

They expect that each person is exercising his or her professional judgment to take the path that he or she believes is best at each juncture of the project, or at least to raise questions and concerns to be discussed with the team, management, and the customer.
But the Agile methods do not just expect motivated individuals to appear. The second sentence of this principle highlights a few conditions that the Agile Manifesto authors believe are important to building a motivated team: an appropriate environment, support for them as professionals, and trust that they will indeed exercise good professional judgment.

These things highlight the Agile methods’ differences from the more conventional command-and-control mode for managing projects. The traditional methods assume that intrinsic motivation is rare, and so they enforce specific behaviors. The Agile methods build an environment where each individual is challenged to build his or her capabilities and where motivation is an expected outcome of the environment.

Perhaps a better way to state this principle would be, “Build project environments that generate motivated individuals.” This statement aligns more readily with the way the Agile methods are engineered.

Self-organizing teams

"The best architectures, requirements, and designs emerge from self-organizing teams."

The Agile methods embrace the recent movement toward self-managed, self-directed, or (as this principle says) self-organizing teams. This philosophy is a movement away from traditional command-and-control management and toward one that counts the team as an entity that has its own knowledge, perspective, motivation, and expertise. In this environment, the team is treated as a partner with management and the customer, capable of providing insight, affecting decisions, and negotiating commitments.

The Agile Manifesto sees this structure not only as a way to achieve a more motivated team but also (as this Principle states) as a way to achieve technical excellence. The Agile methods each count on the team itself to keep the technical issues under control and “on the radar scope” when project decisions are being made.

Different agile methodologies suggest different practices to start and provide required environment in order to make motivated individuals and self directed teams.

Adaptive Software Development suggest Development team as an independent agent that should be involved with other agent such as management and customer. and Speculate for Project initiation and adaptive planning and The Adaptive (Leadership-Collaboration) Management Model

Dynamic System Development Method says that DSDM teams must be empowered to make
decisions

XP suggest The Planning Game and Collective ownership

Feature-Driven Development (FDD) suggest Class (code) ownership and Feature teams

Lean Software Development suggest Self-determination and Motivation

And finally Scrum says that The members of the Scrum development team are key participants in Sprint (development cycle) planning in a Scrum project. They participate fully along with other stakeholders in planning each Sprint, and their commitment
to achieving the Sprint goals is a key criterion for an acceptable Sprint plan.

During each sprint, the Scrum team has full authority to do whatever they believe is necessary to achieve the Sprint goals. They even have the authority to abort the Sprint if they discover that the agreed-upon goals cannot be met. Thus, the Scrum team self-manages and (it is expected) also self-motivates.

But the basic philosophies of these methods are remarkably similar, here is brief description:

Trusting the technical team
Abandonment of command-andcontrol style management; replacing it with clear respect for the team:

◗ Include the team in all deliberations about the project.
◗ Give them unfettered access to the customer.
◗ Seek their input on all technical issues.
◗ Believe their concerns about schedule goals.
◗ Expect them to learn (along with everyone else) as the project progresses.
◗ Gain their willing commitment to all plans.
◗ Negotiate with them to reach acceptable plans.
◗ Empower them with broad authority.
◗ Provide the tools and support they need.
◗ Trust them to exercise their best professional judgment in all circumstances.

This essentially means entrusting the development team with the authority to self-organize and self-manage.

Staffing with “motivated individuals”

You may be asking yourself, “Where am I going to find all of these motivated people?” “And what will I do with the unmotivated lumps I am stuck with now?”

It is true; most of your staff members are not superstars. In fact, most of them are pretty average. The point behind the Agile methods is not that you must find a whole bevy of highly motivated and technically expert people. Rather, it is that by adopting an Agile Method and what ASD calls the “leadership-collaboration” management style, you can motivate most of your current staff to behave more like superstars.

Behavioral psychology has long told us that people tend to live up to (or down to) the expectations placed on them. This is as true with your technical staff as with any other population. The average person, in an environment where his or her expertise and knowledge is valued, will not only bring those attributes to bear on the topic at hand but will also seek to improve them, to make himself or herself more capable of rising to such
challenges in the future.

Pair Programming
XP’s practice of “Pair Programming” is unique and foreign to almost all organizations. It states that pairs of individuals working together perform all technical work. These pairs are expected to form and trade partners from time to time. All designing, test case development, coding, and testing is done in pairs. While one member of a pair is “driving” the computer, the other is watching, assessing the partner’s work, and asking questions.

The pair switches roles on a regular basis. Although this practice sounds wasteful of your most precious resource (your engineers’ time), XP’s proponents claim just the opposite. They claim that a pair working together can be as much as 50% more productive than the two individuals would be if they worked separately. This is expected for a variety of reasons:

◗ The real-time review being done by the observing partner results in many defects being discovered and corrected within minutes of being created, instead of showing up later in tests when the diagnosis and correction could be more time-consuming.

◗ The person in the observer role tends to think about the big picture, making him or her more likely to identify architecture, design, or testing issues early, so they can be corrected before much rework would be required.

◗ The constant interaction between the individuals allows them to learn from each other, sharpening both of their skills and making each of them more capable.

◗ Because two individuals become intimately familiar with every part of the system, the loss of any one person cannot cripple the project.

◗ It provides a way for any new project member (even a recruit fresh out of school) to become a contributing member of the project very quickly.

Chief Programmer
FDD includes a Chief Programmer role. This role is generally reserved for highly respected team members who have superior technical knowledge. By placing such individuals into this role, FDD seeks to capitalize on their expertise and reputation to strengthen the skills of the rest of the team. In addition to their normal technical tasks, Chief Programmers act as coaches and mentors on technical matters, being a resource to team members who have questions about the work they are contemplating.

Method Coach
Some Agile methods (e.g., XP and Scrum) recommend that the project have a person whose job is to coach the team in using the method. This is a good recommendation in any environment where the team is being asked to change how they do their work. Changing old habits can be difficult, and a coach may be indispensable in helping people recognize old habits and exchange them for new ones.

A coach can keep things running efficiently by being the person who is always concerned about the method and how well it is serving the project’s needs. And, of course, when team members do become aware of the method (because it is impeding their work), the coach will be prepared to help them work through the problem and come up with ways of working that serve the team better. The Coach ensures that the Method you have adopted is indeed creating a project environment in which the team can self-motivate and self-manage.

People over processes and tools Or Balancing people, process, and tools

2010-01-10T03:07:00.000-08:00

The very first value in the Agile Manifesto draws a line in the sand. In it, the Agilists clearly state their belief that people are of greater importance in determining software project success than are processes or tools.

The software process community generally acknowledges the importance of people, but the processes they build do not always reflect such a belief. Like the toolmakers, the writers of processes often focus so closely on the completeness and robustness of the processes themselves that the ability of people to follow them is compromised. For example, a person may find that the role he or she is assigned by the process may require activities beyond his or her skills, abilities, or time constraints.

Compounding this problem, when people cannot see the value of the work products or activities that a process prescribes, they will consciously or unconsciously undermine or circumvent that process.

Process writers also tend to pay little attention to tools. While they acknowledge that the right tools can make any process more efficient, they do not routinely consider the available tools when designing processes. In light of this discussion about the effects of tools on process, such a lack of attention can have serious negative effects as the organization attempts to integrate a new process with an incompatible tool set.

So we see that of people, process, and tools, all are important to our projects’ success. they are the three legs on which projects stand. To make one leg longer or shorter than the other two would make the project unstable, and eliminating any of the three would cause the project to fail. The success of our projects depends on all three.

The role of people

Our people are our most precious resource. In the business of creating intellectual property, people have a preeminent role.

Processes cannot create. Tools cannot exercise intellect. Turning ideas into working software requires people. People imagine, people interpret, people envision what they expect to build, and then people turn that vision into reality. Without people, software cannot be written.

but people have shortcomings. People make mistakes that result in defective software. People forget things so that their solutions are incomplete. People envision the future imprecisely so that their plans and estimates may be poor. People can keep only a limited amount of information in their minds at one time, so they may miss the consequences of their decisions. People misinterpret what others say so that information is lost in communication. People remember imprecisely so that facts are subject to dispute. People are expensive and may work slowly so that projects run over budget and schedule. People are bored by repetition so that some tasks are ignored or performed poorly.

Because of all these things and more, people by themselves are insufficient to ensure a successful software project. A team of even the best people in the software industry will have an uncertain likelihood of success. All people need the support that is provided by processes and tools to mitigate for their shortcomings so they can do their best work.

The role of processes

Processes identify the roles of the people on the project, the actions those people will take, and the work products those people will produce. In fact, the Agile Manifesto’s contention that interaction among individuals is more important than “process” belies a misunderstanding of the nature of process.

After all, it is the process that is being followed that determines who will interact with whom, under what conditions the interaction will take place, what will be the subject of that interaction, and what will be its result.

Indeed, “interaction” cannot be more important than “process,” because interaction is, itself, part of the process!

The importance of process lies precisely in the shortcomings of people, as we already discussed people make mistakes, so their processes include checks and balances. People forget things, so their processes remind them of what needs to be done. People are imprecise, so their processes identify where precision is needed. And people are limited, so they rely on their processes to keep the important facts before them. Processes that do these things provide the support people need to mitigate for their shortcomings.

Processes are essentially nothing more than tools that mitigate for people’s shortcomings. Good processes make people able to harness their intellectual abilities and effectively apply them to solving problems.

When consistently applied, good processes produce predictable results. And documenting those processes allows multiple people to share them and work together toward common goals.

The role of tools

All software projects use tools. At the very least, they use an interpreter or compiler and linker. Most also use a code control system and something to track bugs.

The main purpose of tools is to make processes more efficient and people more productive. Many labor-intensive tasks can be done more efficiently by either replacing human effort with a tool (such as when we run automated tests) or by supplementing human effort (as is the case when the compiler converts source code we have written into object code).

Repetitive tasks that produce boredom in people (and the resultant errors) can often be relegated to tools. The key to ensuring that a tool is effective lies in making sure that it supports the people who do the work as well as the processes those people use. A tool supports the process when it makes that process less laborintensive and easier to follow.

Balancing people, process, and tools

People, processes, and tools: No one of these is more important than the others. Each has an important role to play in ensuring the success of our projects. None can be eliminated or de-emphasized without having an adverse effect on the project.

Our people are the source of the creativity, intellect, and vision that is required to build software. These key strengths make people indispensable to the task of creating software. But people’s shortcomings can jeopardize our projects’ success. Errors, oversights, miscommunication, and the like are common human frailties. People have come to depend on processes to mitigate for these shortcomings and make success more achievable.

Individuals rarely document their processes and often do not recognize that they follow them. But in organizations, attention to and documentation of process is the key to ensuring that the processes are followed consistently and modified when necessary so they remain beneficial to the organization. Finally, tools make both the people and their processes more efficient by leveraging people’s effort and taking over some of the more mundane (though critical) activities in our processes.

To be sure, there is a clear hierarchy: People require the support of processes, and tools must support both people and processes. But to claim that any one of them is more important than the others is to add unnecessary risk to our projects.

The 12 Principles of Agile Methods

2010-01-01T06:27:00.000-08:00

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Few of us would argue that satisfying the customer is unimportant. The distinctive feature of this principle is the means by which the Agile methods pursue that goal.
“… through early…delivery of…software.” Agile methods tend to shun long upfront requirements development and design activities. Rather, after laying an appropriate foundation (“appropriate” being defined differently for each method), they seek to develop working software as quickly as possible.

In essence, these methods replace the traditional requirements and design phases of development with early proof-of-concept activities. The Agile methods look very much like prototyping projects but with the distinction that the resulting product is delivered for use, instead of being thrown away.

“… through … continuous delivery of … software.” Agile methods all assume an incremental development strategy. They seek to mitigate risk by delivering increments quickly and often (every couple of weeks to every couple of months) so that the customer can validate the project’s decisions and assumptions and verify the utility of the resulting software.
“… through … delivery of valuable software.” Most Agile methods allow the customer to define what is most valuable and give the customer a significant role in establishing the order in which features are delivered.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

This is the core principle on which the Agile methods are built and the reason why the name “Agile” was adopted. This principle acknowledges the fact that change is inevitable and establishes the philosophy that encouraging regular change is an advantage of the Agile methods for their customers. Rather than trying to suppress or control change, the Agile Methods are designed to allow—even encourage—change throughout the project’s life.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter time scale.
This principle expands on the “continuous delivery” phrase from the first principle. It is explicit that the increments on Agile projects should be as short as possible and goes so far as to say that a week or two is not too short for an increment. It also sets an upper limit that might be surprising to many, since development phases of fewer than three months are generally not seen.

Business people and developers must work together daily throughout the project.
The Agile methods all require collaboration between the development team and the other stakeholders in the project. A most crucial role in Agile projects is given to the customer (or the ultimate user of the system being developed). But all other stakeholders (identified by these technophiles as “business people”) have important roles and are expected to interact with the team on a regular basis.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
The Agile methods are all built on the assumption that development team members are all competent, motivated, supported by the organization, and empowered to do their jobs. At the same time, each method builds an environment that is intrinsically motivating to the technical staff and has the natural effect of building each staff member’s capability to self-motivate and self-manage.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
This is the principle that explains why the Agile methods tend to have few written documents. They all favor face-to-face communication as the primary means of sharing information, often supplementing it with tools such as whiteboards. The results of communication are often documented in the form of “information radiators,” which are visually available postings of information to which people can refer whenever it is needed.

Working software is the primary measure of progress.
This principle is another way in which the Agile methods strike against documentation. Most software projects will identify the acceptance of a requirements or design specification as a milestone showing significant progress. Many Agile methods (though not all) reject that idea. They tend to place little value on the project’s interim products and focus almost entirely on the resulting software. Of course, since working software is delivered so often on an Agile project (see the third principle), it becomes a reasonable
measure of progress.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
In many organizations, overtime is the norm, and in some it is routinely required. The Agile methods rebel against those norms and argue instead for “sustainable” schedules (generally meaning something in the vicinity of 40 hours per week). And by this they do not mean to plan for 40 hours per week, then work 80; rather, they mean to change the expectation of what can be accomplished so that overtime is rare. But as this principle states, the nature of the Agile methods tends to level out the demands on the project team so that a more sustainable pace is naturally maintained.

Continuous attention to technical excellence and good design enhances agility.
This is perhaps the most valuable principle of the Agile methods. Each Agile method has a strong focus on ensuring that the quality of the technical work is maintained at a high level. The different methods do this in different ways, but each has a strong quality component.

This principle provides a refreshing counterpoint to the all-too-common perception that the developers’ job is to write the software and someone else is responsible for making sure it works. In the Agile methods, development teams take on the primary responsibility to be sure that what is delivered is correct and that the software satisfies the customer’s needs. This focus does not mean that independent verification and validation is unnecessary; rather, it means that the software that is delivered for IV&V will be of higher quality and generally more ready for final testing.

Simplicity—the art of maximizing the amount of work not done is essential.
Before the Agile Manifesto was written, the methods that fall under the “Agile” umbrella were generally referred to as “light” methods. The term “light” was dropped out of recognition that certain application domains require heavier processes (e.g., developing software for safety-critical systems). Even so, the Agile methods always stress using the lightest possible processes.

The best architectures, requirements, and designs emerge from self-organizing teams.
Self-management is a hallmark of Agile methods and a backlash against the command-and-control methods used to manage many software projects. There is growing evidence that self-managed teams are in fact quite effective.

But moving toward self-managed teams requires significant changes throughout the organization. The most significant change occurs in the ranks of management, where managers’ roles must evolve toward coaching and leading, as engineers’ roles grow to include self-management. These changes are difficult to effect and require that many people learn new skills and behaviors, but they can have significant positive results.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
The concept of retrospective or postmortem analysis is well known in the industry (if not widely practiced). The Agile methods adopt this important activity as their primary means of improving their processes. Some Agile methods take this to an extreme by doing these analyses at the end of each increment of development, resulting in lessons learned and process improvements many times each year.

Lean Versus Agile

2009-12-17T06:36:00.000-08:00

Both Lean and Agile software development aim to improve the quality of the software (as perceived by the customer) as well as the productivity of the software development process. They both value (and even welcome) the changes in requirements that will almost certainly occur over the course of the project. They both place the highest value on delivering software that meets the customer’s real needs (not the initial perceived customer needs).

The difference is in the underlying perspective and philosophy…the mindset.

Agile mostly concerns itself with the specific practice of developing software and the project management that surrounds that software development. Agile methods do not generally concern themselves with the surrounding business context in which the software development is taking place.

Lean principles, on the other hand, can be applied to any scope, from the specific practice of developing software to the entire enterprise where software development is just one small part. It is even common in Lean manufacturing to go outside the company and include suppliers in the scope of Lean process improvement. The larger the scope, the larger the potential benefits. Most Lean efforts start out small and expand their scope over time, realizing more and more benefits in the process. In any case, it can be safely said that Lean views all Agile methodologies as valid supporting practices.

The primary focus of Agile software development is on close customer collaboration and the rapid delivery of working software as early as possible. Lean sees that as worthwhile, but its primarily focus is on the elimination of waste in the context of what the customer values.

A key Lean tool for detecting and eliminating waste is the value stream map (VSM). The VSM is a map-like diagram of all activities that take place from beginning to end—for example, from when the customer requests a new feature to when that new feature is delivered to the customer. Each step is then identified as value-added (from the customer’s perspective), non value- added, or non-value-added but necessary.

Finally, Agile has a fair number of formal methodologies, whereas Lean has no formal methodologies. Lean, instead, has a toolkit of recommended practices from which to choose. In fact, when implementing Lean software development, it is quite common to pick a lightweight Agile methodology as a starting point and begin applying other Lean tools (such as VSM) from there.

Lean Software Development

2009-12-17T06:25:00.000-08:00

Eliminate Waste

Defects → defects
This one is easy. A defect is a defect, whether you’re talking about manufacturing a physical product or developing software. Defects cause expensive rework, which is always non-valueadded waste. The focus in a Lean environment is on preventing defects, whereas traditional development focuses on finding defects after they have already occurred. Defects are especially expensive when detected late.

In Lean, when a defect is found, the response is to find the root cause of the defect and make changes that will ensure the defect cannot recur. In software development, this means having a suite of automated tests that prevent defects from slipping into the software undetected. When a defect does slip through, a new test is created to detect that defect so that it cannot pass through undetected again.

Overproduction → extra features
Every line of code costs money. Over the lifetime of the software, the cost of writing the code is probably the smallest cost. The code must also be designed, documented, and maintained (changed, enhanced, debugged, etc.). This means that a cadre of current and future team members must repeatedly read and understand the code. The presence of the code must be taken into account in each future product change or enhancement.

The 80/20 rule applies in most software products: 80% of the user’s real needs are provided by 20% of the product’s features. This means the other 80% of a product’s features are rarely (or never) used. At the XP 2002 Conference, the Standish Group reported that 45% of features were never used and that only 20% of features were used often or always. In fact, a 2001 paper titled “Improving Software Investments through Requirements Validation” (IEEE 26th Software Engineering Workshop) found that in 400 projects examined over a period of 15 years, less than 5% of the code was actually useful or used!

Transportation → handoffs
If you have worked on large projects, this might sound familiar:
1. Analysts create a document containing all of the product’s requirements and hand it off to the architects.
2. Architects take the requirements and create the product design, which they hand off to the programmers.
3. The programmers write the code to implement the design and pass the results to the QA testers.
4. The QA testers validate the resulting product against the requirements.

This is a classic Waterfall process that is replete with handoffs. A tremendous amount of knowledge is lost through each handoff simply because it is not possible to record everything that was learned, discovered, created, and known in a written form. A large amount of tacit knowledge is not passed on.

This means that the architects won’t understand the requirements as deeply as the analysts, and that the programmers will not understand the design as well as the architects. This incomplete understanding will lead to errors and omissions, which will require costly rework to correct.

Even worse, the knowledge loss is compounded with each handoff. For example, if you assume that 50% of the knowledge is lost in each handoff, the programmers only receive 25% of the original knowledge, losing a whopping 75% in only two handoffs! Try to avoid handoffs whenever possible.

Inventory → partially completed work
Simply stated, partially completed work is anything that has been started but not finished. This could be requirements (features) that haven’t been coded, or code that hasn’t been tested, documented, and deployed, or bugs that haven’t been fixed. Rather than letting partially done work build up in queues, the Lean approach uses single-piece flow to take a feature through to deployment as rapidly as possible.
A feature is not complete until it is potentially deployable (other constraints may not allow actual deployment as often as we would like), fully documented, tested, and error-free.

(Over) processing → unneeded processes
Unneeded processes are pure waste. They get in the way of productivity without adding any value. They include procedures that accomplish nothing and documents that no one reads. They also include manual tasks that could be automated and procedures that make simple tasks hard.

Build Quality in
One of Taiichi Ohno’s key manufacturing insights was that you cannot inspect a product for quality at the end of a production line. That approach detects problems, but it does nothing to correct them. Instead, each step in the process should be mistake-proof and self-inspecting.

When a problem is detected, the entire assembly line stops until the root cause of the problem is found and corrected (so that it cannot occur again).

A famous example is the New United Motor Manufacturing, Inc. (NUMMI) automobile factory, a joint venture between Toyota and General Motors. Toyota told workers simply to do good work and to stop the line whenever something prevented them from doing their jobs. It took nearly a month to produce the first vehicle, but since each problem was solved once permanently the plant quickly became a leader in quality and productivity in the U.S.

Traditional software development has followed the same pattern as traditional U.S. car manufacturing: let defects slip through and get caught later by QA inspections. The Lean approach is to mistake-proof your code by writing tests as you code the features. These tests prevent subsequent changes to the code from introducing undetected defects.

Create Knowledge
The point here is not to forget the lessons you have learned. Obviously, making the same mistakes over again or relearning how something works is a waste of time and effort. And it’s not just you, your coworkers shouldn’t have to learn something you have already figured out. Find ways to record your team’s knowledge so that you can easily locate it the next time you need it. It’s hard to be specific about this because what makes sense and what will work for you is highly dependent upon the context. However, it is generally best to store a given piece of knowledge closest to its source.

Defer Commitment
The best decisions are made when you have the most information available. If you don’t have to make a particular decision now, wait until later when you have more knowledge and information. But don’t wait too long, either lack of a decision should not hold up other aspects of the project.

Deliver Fast
Software development is an abstract endeavor. Yet most of us (including our customers) work best when dealing with concrete things. When we can see, touch, and feel something, it becomes real, and our brains can more easily think about what works and what doesn’t. Our imaginations can dream of features or capabilities that we didn’t realize we needed.

This is why software requirements are so volatile. The Waterfall approach would have you waiting until the end of the project to get customer feedback based on the actual use of the software, and that is why the Waterfall process is so prone to failure. “Deliver fast” means developing features in small batches that are delivered to the customer quickly, in short iterations. These features can be implemented and delivered before the associated requirements can change. This means that the customer has an opportunity to use these features (which are now concrete) to provide feedback that can change the other requirements before they are implemented.

The completion of each short iteration provides an opportunity to change and reprioritize the requirements based on real feedback and use. The end result is a product that more closely meets the real needs of the customer, while simultaneously eliminating the tremendous amount of waste and rework created by the requirements churn—truly a win-win situation.

Respect People
This is a lofty altruism that is also the down-home truth. As Mary and Tom Poppendieck said in Implementing Lean Software Development, “Engaged, thinking people provide the most sustainable competitive advantage.”

Respect for people means trusting them to know the best way to do their jobs, engaging them to expose flaws in the current process, and encouraging them to find ways to improve their jobs and the surrounding processes. Respect for people means recognizing them for their accomplishments and actively soliciting their advice. Don’t waste your most valuable resource—the minds of your team members!

Optimize the Whole
This is an important part of Lean thinking no matter where Lean is being applied, and Lean software development is no exception. Whenever you optimize a local process, you are almost always doing so at the expense of the whole value stream (this is suboptimizing). If you don’t have control over the entire value stream, you may be forced to suboptimize a piece of it. In general, though, you should always attempt to include as much of the value stream as you can when you try to optimize a process.

The Lean Success Story

2009-12-17T05:44:00.000-08:00

It is only recently that the Lean principles have been applied to software development. In the beginning it all started with Lean manufacturing (some 40 to 60 years ago, depending on when you start the clock). Of course, it wasn’t called Lean back then. It was the Toyota Production System, or Just-In-Time manufacturing. James Womack, Daniel Jones, and Daniel Roos coined the term “Lean” in their 1990 book, The Machine That Changed the World (Harper Perennial).

Lean is a mindset, a way of thinking about how to deliver value to the customer more quickly by finding and eliminating waste (the impediments to quality and productivity). This philosophy is expressed in a set of principles that have proven remarkably applicable to a wide range of business activities. The Lean principles have been used successfully in areas as diverse as the supply chain, the office, engineering, and (of course) software development.

A Whirlwind History of Lean

To really appreciate the emergence of Lean production and its derivatives, you have to
understand what it was replacing (or competing with): mass production.

Henry Ford popularized mass production (which had itself replaced craft production) with the
assembly-line manufacture of the Model T in 1913. Mass production is used to produce large
quantities of goods at a low unit cost. It divides the manufacturing process into small steps that can be carried out by unskilled workers, and it relies on the use of high-precision machinery and standardized, interchangeable parts.

The drawback of mass production is its inflexibility. Since a mass production assembly line is
so expensive to set up (and difficult to alter), it is only economical if it is going to produce large
quantities of the same thing.

In 1945, in post-war Japan, the president of Toyota Motor Company, Kiichiro Toyoda, said
that the Japanese automobile industry would not survive if it did not “catch up with America
in three years.” This did not happen within three years (the Japanese automobile industry
survived anyway), but it did lead to the creation of the Toyota Production System by Taiichi
Ohno.

Taiichi Ohno realized that the American mass production system would not work in Japan.
The domestic automobile market was too small and there was a significant demand for variety

in automobiles, from small economical cars to larger, luxury cars—a poor fit for mass
production. Out of necessity, Taiichi Ohno experimented with many ideas and techniques that
eventually evolved into the Toyota Production System.

Taiichi Ohno described the Toyota Production System as “a system for the absolute elimination of waste.” He wasn’t kidding. By the early 1990s, Toyota was 60% more productive with 50% fewer defects than its non-Lean competitors. According to Ohno, this striking advantage rested on two pillars: Just-In-Time and autonomation.

Just-In-Time
In the 1950s, a Japanese delegation from Toyota visited American businesses to study their
methods. They visited Ford Motor Company, the industry leader, but they were not impressed.

They were particularly appalled by the large amounts of inventory and the unevenness of the
amount of work performed in different parts of the factory.

However, when they visited an American supermarket, they were impressed with the way in
which products were reordered and restocked only after customers purchased them. This pull system inspired Taiichi Ohno to create Just-In-Time, which strives to keep inventories at each manufacturing step as low as possible (preferably zero). It is about providing the right material, in the right amount, at the right time, and in the right place.

According to Ohno, inventory is waste that costs the company money. Even worse, inventory
hides problems in the production system. This includes problems such as inadequate capacity,
inflexible equipment, and unreliable equipment. A major contribution of a Just-In-Time
system is that it exposes the causes of inventory-keeping so that they can be addressed.

Autonomation (Jidoka)
Autonomation is a combination of the words autonomous and automation. It describes
machines that automate a process but are also intelligent enough to know when something is
wrong and stop immediately. This kind of machine can run unattended (autonomously) while
providing workers with full confidence that it is operating flawlessly. It doesn’t have to be
monitored, and it needs human attention only when it stops.

When it detects an abnormal condition, the machine will stop itself and a worker will halt the
production line. This focuses everyone’s attention on finding the root cause of the problem
and fixing it so that it will not recur. In this way, autonomation prevents the production of
defective components that would otherwise disrupt the production line or result in more costly rework at a later stage.

Waste (Muda)
The overarching goal of Lean production (or any form of Lean) is to deliver value to the
customer more quickly, and the primary way to do this is to find and eliminate waste. On the
surface this may seem like a simple thing, but exactly what is and what is not waste isn’t always obvious. Shigeo Shingo, who codeveloped the Toyota Production System with Taiichi Ohno, identified seven kinds of waste

DOTWIMP: THE SEVEN DEADLY WASTES
Lean has some pretty strict views on waste. Shigeo Shingo identified seven types of waste that are easily remembered by using the acronym DOTWIMP:

Defects
This is perhaps the most obvious type of waste. Lean focuses on preventing defects instead of
the traditional “find and fix” mentality.

Overproduction
Producing more than is needed, or producing it before it is needed. It is often visible as the
storage of materials.

Transportation
The unnecessary movement of parts between processes. When you move material and parts
between factories, work cells, desks, or machines, no value is created.

Waiting
People or parts waiting for the next production step.

Inventory
All material, work-in-progress, and finished products that are not being processed. Inventory
beyond the bare minimum consumes productive floor space and delays the identification of
problems (if you’ve got plenty of spares, there’s no incentive to fix quality-related problems).

Motion
People or equipment moving or walking more than is needed to perform the processing.

Processing
Overprocessing beyond the standard required by the customer. This adds additional cost
without adding additional value.

Additional eighth waste: underutilization of people
This is often cited as an additional type of waste beyond the original seven, and it refers to the
underutilization of the worker’s creativity and resourcefulness.

A key Lean activity is to break down a process into a map of its individual steps and identify
which steps add value and which steps do not—the waste. This is known as a value stream
map. The goal, then, is to eliminate the waste (muda) and improve the value-added steps
(kaizen). The waste is further subdivided into two categories: non-value-added but necessary
(given the current state of the system), and pure waste. Pure waste is easy to deal with—it can be eliminated immediately.

So, there are two key Lean skills: knowing what the customer values, and knowing how to
spot waste.

Lean Principles

Taiichi Ohno started with Just-In-Time and autonomation, the two pillars of the Toyota
Production System. Modern-day Lean has settled on five principles and a wide array of
practices that have been distilled from the Toyota Production System and the experiences of
other companies that have followed Toyota’s lead. These five principles are identified as Value, Value Stream, Flow, Pull, and Perfection:

Value
Value is defined by the customer. What does the customer value in the product? You have
to understand what is and what is not value in the eye of the customer in order to map
the value stream.

Value stream
Once you know what the customer values in your product, you can create a value stream
map that identifies the series of steps required to produce the product. Each step is

categorized as either value-added, non-value-added but necessary, or non-value-added
waste.

Flow
The production process must be designed to flow continuously. If the value chain stops
moving forward (for any reason), waste is occurring.

Pull
Let customer orders pull product (value). This pull cascades back through the value stream
and ensures that nothing is made before it is needed, thus eliminating most in-process
inventory.

Perfection
Strive for perfection by continually identifying and removing waste.

New Features and Enhancements in Spring 3.0

2009-12-17T01:29:00.000-08:00

If you have been using the Spring Framework for some time, you will be aware that Spring has undergone two major revisions: Spring 2.0, released in October 2006, and Spring 2.5, released in November 2007. It is now time for a third overhaul resulting in Spring 3.0.

Java 5

The entire framework code has been revised to take advantage of Java 5 features like generics, varargs and other language improvements. SpringSource done best to still keep the code backwards compatible. and now have consistent use of generic Collections and Maps, consistent use of generic FactoryBeans, and also consistent resolution of bridge methods in the Spring AOP API. Generic ApplicationListeners automatically receive specific event types only. All callback interfaces such as TransactionCallback and HibernateCallback declare a generic result value now. Overall, the Spring core codebase is now freshly revised and optimized for Java 5.

Spring's TaskExecutor abstraction has been updated for close integration with Java 5's java.util.concurrent facilities. Spring provide first-class support for Callables and Futures now, as well as ExecutorService adapters, ThreadFactory integration, etc. This has been aligned with JSR-236 (Concurrency Utilities for Java EE 6) as far as possible. Furthermore, Spring provide support for asynchronous method invocations through the use of the new @Async annotation (or EJB 3.1's @Asynchronous annotation).

Overview of new features

This is a list of new features for Spring 3.0. We will cover these features in more detail later in this section.

• Spring Expression Language
• IoC enhancements/Java based bean metadata
• General-purpose type conversion system and field formatting system
• Object to XML mapping functionality (OXM) moved from Spring Web Services project
• Comprehensive REST support
• @MVC additions
• Declarative model validation
• Early support for Java EE 6
• Embedded database support

Core APIs updated for Java 5

BeanFactory interface returns typed bean instances as far as possible:

• T getBean(Class requiredType)
• T getBean(String name, Class requiredType)
• Map getBeansOfType(Class type)

Spring's TaskExecutor interface now extends java.util.concurrent.Executor:
• extended AsyncTaskExecutor supports standard Callables with Futures

New Java 5 based converter API and SPI:

• stateless ConversionService and Converters
• superseding standard JDK PropertyEditors
Typed ApplicationListener

Spring Expression Language

Spring introduces an expression language which is similar to Unified EL in its syntax but offers significantly more features. The expression language can be used when defining XML and Annotation based bean definitions and also serves as the foundation for expression language support across the Spring portfolio.

The Spring Expression Language was created to provide the Spring community a single, well supported expression language that can be used across all the products in the Spring portfolio. Its language features are driven by the requirements of the projects in the Spring portfolio, including tooling requirements for code completion support within the Eclipse based SpringSource Tool Suite. The following is an example of how the Expression Language can be used to configure some properties of a database setup

This functionality is also available if you prefer to configure your components using annotations:

The Inversion of Control (IoC) container

Some core features from the JavaConfig project have been added to the Spring Framework now. This means that the following annotations are now directly supported:

• @Configuration

• @Bean
• @DependsOn
• @Primary
• @Lazy
• @Import
• @ImportResource
• @Value

Here is an example of a Java class providing basic configuration using the new JavaConfig features:

To get this to work you need to add the following component scanning entry in your minimal application context XML file.

Or you can bootstrap a @Configuration class directly using AnnotationConfigApplicationContext:

Defining bean metadata within components
@Bean annotated methods are also supported inside Spring components. They contribute a factory bean definition to the container.

General purpose type conversion system and field formatting system
A general purpose type conversion system has been introduced. The system is currently used by SpEL for type conversion, and may also be used by a Spring Container and DataBinder when binding bean property values.
In addition, a formatter SPI has been introduced for formatting field values.

This SPI provides a simpler and more robust alternative to JavaBean PropertyEditors for use in client environments such as Spring MVC.

The Data Tier

Object to XML mapping functionality (OXM) from the Spring Web Services project has been moved to the core Spring Framework now. The functionality is found in the org.springframework.oxm package

The Web Tier

The most exciting new feature for the Web Tier is the support for building RESTful web services and web applications. There are also some new annotations that can be used in any web application. Comprehensive REST support Server-side support for building RESTful applications has been provided as an extension of the existing annotation driven MVC web framework.

Client-side support is provided by the RestTemplate class in the spirit of other template classes such as JdbcTemplate and JmsTemplate. Both server and client side REST functionality make use of HttpConverters to facilitate the conversion between objects and their representation in HTTP requests and responses.

The MarshallingHttpMessageConverter uses the Object to XML mapping functionality mentioned earlier.

@MVC additions
A mvc namespace has been introduced that greatly simplifies Spring MVC configuration. Additional annotations such as @CookieValue and @RequestHeaders have been added.

Declarative model validation
Several validation enhancements, including JSR 303 support that uses Hibernate Validator as the default provider.

Early support for Java EE 6
Spring provide support for asynchronous method invocations through the use of the new @Async annotation (or EJB 3.1's @Asynchronous annotation).
JSR 303, JSF 2.0, JPA 2.0, etc

Support for embedded databases
Convenient support for embedded Java database engines, including HSQL, H2, and Derby, is now provided.

What's New in Java EE 6

2009-12-14T05:43:00.000-08:00

Java EE 6 just got released yesterday, here is everything you need to know about, in brief.

A look back: Java EE 5 was a remarkable step towards a mature, widely deployed, well supported server-side development platform. EJB 3.0 was re-engineered to ease, the Entity Bean model was stripped and replaced by JPA as the persistence paradigm, JSF was introduced as the standard presentation tier framework and JAX-WS 2.0 replaced JAX-RPC as the SOAP web services API. The focus of Java EE 5 was squarely on reducing complexity by embracing the concepts of annotations, POJO programming, zero-configuration systems.

Overview

Java EE 6 is next big step in the journey towards the idea of a “simple, streamlined and well-integrated platform”. Java EE 6 includes a rich paradigm of innovations best reflected in the technologies that comprise the platform including brand new APIs like WebBeans 1.0 and JAX-RS 1.1, JPA 2.0 or even mature APIs like Servlet 3.0, EJB 3.1

JSR 316: JavaTM Platform, Enterprise Edition 6 (Java EE 6) Specification.

Let’s give each one of the features a brief look:

Pruning

Pruning the Dead meat: The Java EE platform is gainging weight since it’s initial launch in 1999. The idea now is to cut the dead (least used) stuff off. Java EE 6 begins the essential process of carefully pruning APIs to make the platform more lightweight and to make room for healthier growth. Few APIs that are chopped off and why:

JAX-RPC — JAX-RPC was an early attempt at modeling SOAP web services as RPC calls. Web services are now much more robust, feature-rich and popular JAX-WS API effectively supercedes JAX-RPC.
JAXR – JAXR is for interfacing with UDDI registries. Unfortunately, UDDI is not widely used.
EJB 2.x Entity Beans CMP — The complex, heavyweight model replaced by the popular, lightweight, POJO based JPA persistence model.
Java EE Application Deployment — JSR 88 was an attempt at developing deployment tools that work across application servers. Unfortunately, this API has never got popular, and never gained much vendor support.
Java EE Management – Application server management tools that work in a cross-vendor manner. Like JSR 88, the idea never paved off.

“Pruning” an API essentially means that an application server vendor need not support these technologies, what this doesn’t mean is that you cannot support those APIs, few larger vendors will probably continue supporting them atleast for some time.

What is the biggest major criticism of Java EE? Its simply too large. The fact is, majority of Java web applications do not utilize/leverage the full Java EE stack. So why is that stuff still in there?

Profiles are designed to address this issue. Profiles are essentially sub-sets of Java EE APIs geared towards a particular class of applications in mind. For example, the proposed Java EE Web Profile will only include APIs that are likely to be used in most Java web applications. The idea isn’t new, Java ME supports the idea of Profiles already.

Although Java EE 6 defines the rules for creating new Profiles through separate JSRs, only one Profile, the Web Profile is included in platform this time. Detailed chart of Profiles can be found at Sun’s documentation.

Web Beans 1.0:

WebBeans is perhaps the most groundbreaking API developed in the Java EE 6 time-frame. WebBeans fills a number of gaps in Java EE. Although WebBeans is inspired by Seam, Google Guice as well as Spring, it does not directly mirror any of them. Indeed, WebBeans adds a number of unique innovations of its own. Here’s brief on it:

WebBeans unifies the JSF, JPA and EJB 3 programming models to truly feel like a single, well-integrated platform.
WebBeans implicitly manages the life-cycle of all registered components in appropriate contexts.
WebBeans brings a robust set of dependency injection features to the platform in a completely type-safe and Java-centric way.
WebBeans enhances the Java EE Interceptor model by adding the ability to bind interceptors to annotations instead of having to bind interceptors to target object classes themselves.

WebBeans adds a lot of other very cool features that pave way to nextgen web applications.

Java EE 6 has arrived!

2009-12-10T12:37:00.000-08:00

Today is the official release day for Java EE 6. It has been almost 10 years since the initial release of J2EE 1.2 which was released on December 17th, 1999.

Here is Java EE 6 Tutorial, Volume I
http://java.sun.com/javaee/6/docs/tutorial/doc/

The Java EE 6 platform includes the following new features:

Profiles, configurations of the Java EE platform targeted at specific classes of applications. Specifically, the Java EE 6 platform introduces a Web Profile targeted at web applications, as well as a Full Profile that contains all Java EE technologies.
New technologies, including the following:
- Java API for RESTful Web Services (JAX-RS)
- Contexts and Dependency Injection for the Java EE Platform (JSR-299), informally known as Web Beans
- Java Authentication Service Provider Interface for Containers (JASPIC)
New features for Enterprise JavaBeans^TM (EJB^TM) components
New features for servlets
New features for JavaServer^TM Faces components

Envisioning Next Generation of Web - The Semantic Web

2009-12-03T08:14:00.000-08:00

The Semantic Web is an extension to the current Web, and started out as a vision of one of its creators, Tim Berners-Lee. He realized that one of the major shortcomings of the traditional Web is the fact that its information is mostly human-interpretable, not machine-interpretable.

To fully grasp this problem, consider the example of below picture, showing two very simple Web pages, each representing a mouse. You, the reader, will immediately notice that on the left Web page an animal is shown, and on the right page a computer accessory is shown. You do so by interpreting the image and using this knowledge to attach the correct interpretation to the word mouse in the sentence This is a mouse." In other words, the meaning or semantics of the word mouse is provided by you, the human reader, using contextual information (in this case, the accompanying image). For a machine, however, it is impossible to grasp what is on each page. In fact, a machine only detects two pages, each containing an image and a string. It does not know what is represented in the image; it cannot interpret the sentence, nor can it correctly distinguish and interpret the two occurrences of the word mouse." For a computer, the information contained in the page is readable, but not interpretable.

This problem is illustrated by the fact that all major search engines are mainly restricted to syntax-based search queries, as they are not able to grasp the semantics of the information on the Web. Also, for automated agents crawling the Web searching for relevant information, the lack of semantics is problematic. Imagine a shopping agent that searches for the cheapest 17-inch screen available. It will run into all kinds of problems related to the lack of semantics: when is something a (computer) screen, when are monitors also

screens, how different currencies are handled, what about promotions? The automated agent will fail miserably.

To overcome this problem of lack of semantics on the Web, Tim Berners- Lee envisioned the Semantic Web. He defined it as an extension of the current Web in which information is given a well defined meaning, better enabling computers and people to work in cooperation . The Semantic Web is thus considered as an additional layer on top of the current Web. To give information a well-defined meaning, the Semantic Web

relies on the use of ontologies. Intuitively, we can describe an ontology as the terminology used for concepts of a certain domain (e.g. mouse, cat, animal, reptile), and the relations that exist between these concepts (e.g., is a, eats). Our enhanced example page, representing a mouse with an ontology, is (schematically) depicted in this picture:

More formally, the term ontology is defined as a formal, explicit specification of a shared conceptualization of a domain of interest. In other words, ontologies form a formal and explicit representation of concepts and the relations between these concepts that can be distinguished in a particular domain. This explicit and formal specification allows machines to understand and interpret the information captured by an ontology, thus allowing automated processes to use this information (e.g. for reasoning, for crawling the Web, for performing search based on content).

Learning Organization in Practice - Senge’s View "The Fifth Discipline"

2009-11-28T06:20:00.000-08:00

1. Personal Mastery

Personal mastery is concerned with individuals’ desire and ability to grow through learning. Those who have the discipline of personal mastery are actually ‘masters of personal learning’; people who value learning and who act on what they learn.

This kind of mastery implies a more active form of learning, a desire to find out more and to act on what we find out, turning it into knowledge, and in the process improving our lives.

Achieving true mastery involves a constant search for priorities. Prioritizing is itself a form of learning, which requires awareness, honesty and continual questioning. We must be prepared to note changes in ourselves and our environment, and re-prioritize accordingly.

Learning organizations are built by people with personal mastery. Without people who will question and re-prioritise, any organization risks pursuing the wrong goals and constantly playing catch-up with the opposition. In return, organizations need to value those who value learning. Companies need to provide opportunities to learn and act, and they need to seek ways to align the individual’s goals and objectives with those of the firm.

According to Senge, not only will those who possess mastery learn faster, but they will be more committed and more prepared to take the initiative. Enlightened companies will seize upon these attributes and harness them to the firm’s goals. This can only be a win–win situation: individuals are given opportunities to learn and are happier in their work. Meanwhile, the firm reaps the returns from new ideas, commitment and improved operations.

2. Shared Vision

Sharing a common goal is what distinguishes a team from a group of people. Simply putting people in a room and telling them that they are a team doesn’t make them a team. Similarly, hiring six developers who have worked well in six different teams before and calling them a team doesn’t make them a team. Each team is different, because each team contains a different mix of people and each team needs to find its own way of working together, its own strengths and weaknesses.

Participating in the shared vision means that individuals accept that they won’t create the whole themselves. Instead, individuals are offered the opportunity to create something bigger than they could create alone, and something that will be bigger than the sum of its parts.

For this to happen, each individual must hold a personal stake in the vision. In order to build a personal stake, it helps if each person is allowed to take part in the vision formation. This will help them internalize the vision and make it part of their identity, thus creating a sense of belonging and focus for actions.

Simply handing a team a ready-made vision and asking them to adopt it as theirownis the antithesis of true shared vision. People who are handed a ready made vision won’t feel the same attachment, and consequently they will be less motivated to bring the vision to reality.

For a shared vision or mission statement to really motivate individuals and teams, they must be able to associate closely with the idea. It is not enough for a manager or planner to explain how vision affects an individual. In order to create a close personal relationship with the vision, individuals need to hold a stake in it.

3. Team Learning

In any development team, many members will have specialist skills and knowledge. For the team to work well together, the members need to respect one another and maximize individual’s specialities, but also support the needs of those with less experience or knowledge.

For Senge, there’s more to team learning than just ensuring that a team works well together. Team learning goes beyond good teamwork. Team learning builds personal mastery and shared vision, to create highly effective motivated units.

Team action and learning are guided by the shared vision that helps the team to direct its thinking. The shared vision helps the team to know what to investigate and what to pass over.

When a team works well together, guided by a shared vision that all members believe in and work towards, then a phenomenon of alignment is seen. This occurs when all team members are pulling in exactly the same direction, without diversion. Unfortunately, there’s a danger that effective individuals working with the best intention but not working in alignment can pull in different directions. This makes management more difficult.

Team members need personal mastery to keep their individual learning moving, but for team learning there needs to be more. There needs to be sharing of learning amongst team members and between teams. Individuals can be great learners themselves, but if they don’t share their learning then team learning won’t occur.

Effective teams are open to learning as a group. The team needs to be able to engage shared reflection and what Senge calls dialogue and discussion. Senge differentiates between dialogue – which he considers a creative exploration and discussion, which he describes as the presentation and defence of different views.

The team should inquire into opportunities and problems, and recognize potential conflicts rather than avoiding them. These mechanisms help the team learn, build team knowledge and identify improvements to workpractices.

4. Mental Models

Mental models are the ideas, preconceptions and assumptions that we all carry around in our heads.We need these models to help us function every day, and they are valuable short cuts that enable us to skip over first principles and get things done.

When we’re aware of these models, there’s little problem. If our assumptions are out in the open, are explicit and we’re aware of them, then we can selectively switch them off when dealing with new problems or seeking new solutions. Problems set in when we’re unaware that the models are governing our actions, or when we fail to recognize that our models don’t apply.

Senge goes further than just pointing out that mental models exist. He advocates a discipline that seeks to expose the models and open them to questioning. Recognizing that these assumptions exist allows them to be questioned and rethought.

Mental models exist throughout the software development process. They exist in specification documents, in manager’s models of how development works, in the code that developers write and in the tests run against the system. Some inaccurate assumptions lead directly to bugs found by customers.

At every stage in the development process, we use our mental models to make assumptions. Advocates of voluminous documentation and strict methodological processes can claim that documentation will help overcome the problem. But as we write longer documents, and add extra rigour to our processes, we encourage individuals to take short cuts and rely on their mental models all the more.

5. Systems Thinking

Systems are not just computer systems. They are any kind of device or process where a number of different pieces need to work together to produce an end result or product. Computer systems are just one type of system, namely the type relating to computers. Modern society and business are full of systems.

Traditional Western science emphasizes breaking things down into small parts and understanding the operation of each small piece. We consider substances made up of atoms, and atoms made up of electrons, protons and neutrons; protons in turn are made up of quarks and so on. Systems thinking takes the opposite approach. It seeks to understand how larger entities interact and inter-operate. Rather than looking at individual actions and explaining how each one came to occur we look at the overall system and try to understand the drivers for whole systems.

Issues arise because although all the individual pieces may be working faultlessly, when they work together in the system unintended results occur. Identifying the problems can be difficult because our training, as engineers and scientists, leads us to decompose the problem and examine individual pieces. To overcome this, we need to engage in systems thinking.

Diagnosing problems is only half the story. Once identified, these problems need to be addressed. Again, the complex interaction between multiple pieces makes solutions harder. Fixing a problem may require several coordinated changes. What may appear to be a retrograde step in one piece may actually resolve a far larger problem.

And Reflection

‘‘Skills of reflection concern slowing down our own thinking processes so that we can become aware of how we form our mental models and the ways they influence our actions. Inquiry skills concern how we operate in face-toface interactions with others, especially in dealing with complex and conflicting issues.’’ Peter Senge (1990)

In practice, reflection simply means ‘taking time to think about things’. It is very easy in a hectic environment to be so concerned with getting stuff done that we never stop to think about what we’re doing, why we’re doing it and whether we may actually be making everything worse with the fixes we apply.

Reflection is key to personal growth and development. Actively practising reflection can substantially improve our own learning process and help us achieve our aims and objectives. Team reflection can also help with enhancing team learning.

What is The Open Source Business Model?

2009-11-26T12:59:00.000-08:00

It is often confusing to people to learn that an open source company may give its products away for free or for a minimal cost.

How do open source companies make money?

While it is true that an open source business may not make money directly from its products, it is untrue that open source companies do not generate stable and scalable revenue streams.

In actuality, in the 21st century web technology market, it is the open source company that has the greatest long-term strategic advantage. This is demonstrated by companies such as LINUX, Apache, and Netscape, a host of web-specific technologies such as Java, Perl, TCL, and a host of web-specific technology companies such as Sendmail.

The open source business model relies on shifting the commercial value away from the actual products and generating revenue from the 'Product Halo,' or ancillary services like systems integration, support, tutorials and documentation.)

This focus on the product halo is rooted in the firm understanding that in the real-world, the value of software lies in the value-added services of the product halo and not in the product or any intellectual property that the product represents.

In actuality, the value of software products approaches zero in the fast-paced, highly-customized, ever-changing world of information technology.

But it is not simply an acknowledgement of the revenue streams generated by the product halo that makes open source a compelling business strategy.

Open source also cuts down on essential research and development costs while at the same time speeding up delivery of new products.

This paradoxical situation arises from the fact that within an open source project, the community members themselves provide free research and development by contributing new solutions, features, and ideas back to the community as a whole. The company that sits at the center of any successful open source project may reap the rewards of the work of thousands of highly-skilled developers without paying them a cent.

A final strength of the open source business model lies in its ability to market itself.

Because open source products are typically released for free, open source companies that can produce quality products and generate a good reputation can almost immediately grab huge shares of any market based on the complex and far-reaching global referral networks generated by users.

In fact, in the web technology space, almost every global standard has been based upon open source technology.

By using the open source technology model, we can create a superior product, which immediately has a competitive advantage, and which generates multiple scalable revenue streams while being freely available throughout the community.

Another Perspective – Tests as a shared language

2009-11-15T01:35:00.000-08:00

One of the biggest problems of developing software for someone else is the prevalent ambiguity in requirements. It is far from child’s play to express and communicate requirements in such a way that no information is lost and that the original idea is transmitted fully intact. Some would go as far as saying it’s impossible to do so. After all, we can’t read minds.

This problem is highlighted when the medium used for this communication is written documentation—a requirements specification, for example—which is far from being a perfect way to transfer information and understanding. If we were able to transform the requirements into executable tests that verify whether the system conforms to each particular aspect of the specification, there would be many fewer problems with ambiguity and less room for interpretation on behalf of developers. This is the wonderful premise of tests as specification.

Tests as specification
Tests as specification provides a way to look at tests as something that’s essentially derived from requirements; therefore—in theory at least—a system that passes the tests should by definition conform to the specification. This, however, assumes perfect test coverage, which is rarely—if ever—the case in the so-called “real world” software projects.

Defects get through testing every now and then. That is hardly news to anyone who’s lived through a couple of commercial software projects. In part, this is due to our having missed a test that we should’ve considered. In part, this is due to our human nature and our deceptive intuition, which manages to talk us out of running some of the tests to “save time.”

This begs the question, are we really better off using tests as specification if the tests we have tend to leak in one way or another? Is it really feasible to use tests as specification, effectively redefining the meaning of these concepts? Using tests as specification is not a silver bullet either, but it does have a number of clear advantages that makes it worth considering:
■ Faster feedback through automation
■ More reliable test execution
■ One less translation step

First of all, there’s no denying that having automated, executable tests would get rid of a lot of grunt work and would accelerate our feedback loop enormously compared to manually executed test cases.

Second, executable tests would effectively prevent our inherent laziness from getting us into trouble, because we would be able to exploit the fact that the computer doesn’t suffer from the same problems of character that we humans are born with. Finally, we are already making the same errors in translating the true requirements into writing with our current practice, so would it really be any worse if we’d just change the format and medium we’re using to document our test cases?

There are still plenty of opportunities to err while transforming the requirements into tests, of course, but the less knowledge transfer and human interpretation is involved in running the tests, the less chance there is that something falls into the gaps after the test has been crafted.

Specification by example
Furthermore, one of the most striking benefits of using tests as the specification driving development is that tests typically employ specification by example instead of abstraction or prose (although there should almost invariably be some amount of prose involved—things are not black and white in this respect either).

In other words, instead of the traditional “the system shall calculate tax” pattern familiar in requirements documents, specification by example promotes expressing the requirement through examples such as “for a $20 subscription with a tax rate of 10%, the system charges a total of $22 from the user’s account.”

For this simple requirement, the difference between the traditional “system shall” and an example-based version might not be that significant—after all, we all know how to calculate tax in such a trivial case. However, the requirements we meet at work are typically not as trivial, and the risk of misunderstanding is much higher. For instance, applying multiple taxes for a given transaction might need to employ different calculation rules in different locales, which can be clarified enormously through concrete examples.

Specification by example is a natural fit for our intuition and makes it easier to relate the requirements to the concrete world and our software. Specification by example can also be seen in TDD. Whereas acceptance tests specify by example the desired behavior of the system, the examples and the desired behavior specified with unit tests are specifications about the desired implementation rather than about the functionality delivered.

Higher quality inside and out, better confidence in our work, and the customer loving us for giving them software that meets their needs—who wouldn’t like these improvements, especially if it’s not just talk? After all, in theory, anything and everything is possible.

Software Development with Close collaboration

2009-11-15T01:20:00.000-08:00

Close collaboration is essential for any complex endeavors involving people, and software development with acceptance TDD is no exception. In practice, we want to have an integrated project team instead of separate development, business analysis, and testing teams, let alone a distant QA department.

The fundamental idea is that the way to achieve the best possible level of productivity for the whole team is to nurture rapid feedback and effective, face-to-face communication around concrete, working software instead of passing around test plans, requirements specifications, and bug reports between customers, testers, business analysts, and developers. With acceptance TDD, we are able to collaborate effectively by bringing together the knowledge, skills, and abilities required for doing a good job.

Let’s see how this close collaboration helps us deliver the right product by improving our interactions and reducing the likelihood of misunderstood requirements and costly rework.

Seeing concrete, working software Very few customers have been completely satisfied with what the contractor delivers to them after several months of silence. By feeding the customer with completed functionality as a continuous stream, we’re making sure that, if there’s something wrong or missing, we’ll know about it right away.

This early feedback reduces our risks and costs. Furthermore, by not keeping an inventory of supposedly “completed” items that the customer hasn’t seen yet, we’re avoiding the usual illusion of progress that is built on assumptions and that is supported by document-driven methods and meaningless statements such as “development is 90% complete.”

Building trust and confidence
Another significant benefit of delivering working software early and often is that we’re building trust and confidence both between the team and the customer and within the team itself. By showing the customer (and ourselves) that we can deliver, we’re making life a lot easier for our whole team.

Customer in control
There’s one key difference in the role and power of the customer between incremental development and the traditional waterfall, big design, up-front style of development. With incremental development, the customer gets to choose which features they get first. Similarly, they also get to choose which features are dropped if the team is not able to implement all of them within the project’s allocated time or budget.

The customer’s decisions are, of course, influenced by the cost of the features, which the developers estimate including the cost of delaying, the cost of building things in a different order, and so forth.

The ability of the customer to decide what their money is spent on can really change the way they look at software projects. Finally, the customer is in control of what their money is spent on. Talk about motivating the customer!

Evolving a shared language By encouraging close collaboration between testers, developers, and customers, we are effectively facilitating a situation where the information needed is obtained as quickly as possible—in all directions. Furthermore, this continuous exposure within the team is likely to increase the efficiency of communication as people get to know each other better and begin to evolve a shared language. Software development is a people business, after all, and we shouldn’t neglect that fact.

Keeping code healthy with refactoring

2009-11-15T01:08:00.000-08:00

Proceeding in small increments means that we’re constantly extending the system to do something that its current design might not support. Consequently, we’re constantly extending the system’s design in ways that might break existing concepts as well as introduce new ones.

This, in turn, means that we’re bound to end up with a broken design that’s inconsistent, unbalanced, difficult to understand, difficult to extend, or otherwise having a bad hair day. If we’re out of luck, all of them. And that would seriously hamper our ability to keep delivering software to our customers. Not to worry, though. There is a way to practice evolutionary design without letting the design rot—that way is called refactoring.

Quoting Martin Fowler, the author of Refactoring: Improving the Design of Existing Code (Addison-Wesley, 1999), refactoring is “a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.” That description manages to pack a lot of information into such a short sentence. Let’s spell it out and see what it’s actually telling us about refactoring, shall we?

Refactoring is disciplined
When refactoring (verb), we are not just altering the code’s structure but improving the design by altering it in a controlled manner—by applying small behavior preserving transformations that are called refactorings (noun).

In other words, refactoring is about applying refactorings on code in a controlled manner. This restructuring can be a significant change to the existing design, but it is always performed in small steps, verifying at each stage that the little transformations we’ve made have not changed existing behavior.

We don’t just change code. We first identify specific problems in the design, and then we select appropriate refactorings and apply them carefully and thoughtfully. We wait until a problem begins to present itself, and only then do we solve it. We don’t predict design problems beforehand and prepare for them—that would increase the possibility of creating more problems with our system’s design than solving them.

Refactorings are transformations
A refactoring can also be thought of as a transformation between two states. The starting state has some characteristic you’d like to get rid of or otherwise improve on, and the target state represents a design that would incorporate that improvement. Below picture shows an example of a refactoring called Replace Inheritance with Delegation (also documented in Fowler’s book) that, as its name implies, moves our design from an inheritance-based solution to a delegation based solution.

The reason for doing this refactoring might be, for instance, that the subclass is only extending the superclass in order to reuse a small part of its functionality and, as an unwanted side effect, inherits a whole bunch of data and functionality we don’t care for.

Some of these refactorings are so well defined that modern development tools have automated them. This automation has made refactoring evolutionary designs&feasible for applications and systems of pretty much any size and complexity.

Refactorings alter internal structure
So these transformations are applied to the system’s internal structure—the code—which means that many of the refactorings are very low-level. For example, one of the most common refactorings is called rename method. Renaming a method or a local variable might not seem like too significant a change in the system’s design, but renaming a method from something ambiguous to something clear and concise can make a world of difference to someone new to the code and needing to understand the existing code.

Also, such low-level refactorings are the fundamental building blocks to achieving larger refactorings. These “larger” refactorings are typically those that deal with moving the responsibilities around in your code, introducing or removing an inheritance hierarchy, or making other similar changes that (usually) involve more than one class. In the end, all refactorings can be reduced into a series of smaller steps, such as renaming a variable; adding a new, empty class; changing the return type of a method; and so forth.

Although technically the most common refactorings are things like renaming a method, the number-one reason for refactoring is duplication. It might be duplication in terms of having two similar pieces of code in two methods or classes— something that could be extracted into a single method invoked in both places— but the duplication could also be in terms of responsibilities.

Duplication, of course, is bad for a code base because it makes the system more difficult to
change. Having logic and responsibility duplicated in multiple places is a guaranteed way to introduce defects due to us forgetting to make the change in all necessary corners of the code base.

In addition to not introducing defects with our changes to the code’s internal structure, we also don’t want our refactorings to add functionality. That is, refactorings should preserve behavior.

Refactorings preserve behavior
The latter part of the earlier quote by Martin Fowler says, “without changing [code’s] external behavior.” What does that mean? It means that whatever transformations you apply to the existing code, those transformations should only affect the code’s design and structure—not its externally visible behavior or functionality. In other words, client code that uses the code you’re refactoring should not notice any difference in how the refactored code behaves.

Renaming a method that’s part of a class’s public interface will obviously have to ripple through to client code as well, but the behavior of that method should not change in any way. Similarly, restructuring the code behind the public interface should not in any way change the behavior visible through the public interface.

So we think we’ve been successful at altering internal structure without changing external behavior. Cool. But how can we be sure that our refactorings haven’t changed the code’s external behavior? By running tests against our code, of course! We want to keep our software working, and tests are the tool for ensuring that.

Only ever write code to fix a failing test

2009-11-15T00:28:00.000-08:00

Test Driven Development, is a programming technique based on a very simple rule:
Only ever write code to fix a failing test

In other words, write the test first, and only then write the code that makes it pass.This rule is controversial to many of us who have been schooled to first produce a thorough design, then implement the design, and finally test our software in order to find all those bugs we’ve injected during implementation. TDD turns this cycle around Test first, then code, and design afterward.

Does the thought of “designing afterward” feels awkward? That’s only natural. It’s not the same kind of design we’re used to in the traditional design-code-test process. In fact, it’s such a different beast that we’ve given it a different name, too. We call it refactoring to better communicate that the last step is about transforming the current design toward a better design.

Red-green-refactor is an alternative mnemonic for the TDD cycle of writing a test, making it pass, and making it pretty. What’s with the colors?

When we begin the TDD cycle by writing a test, it fails. It fails because our system is broken right now; it doesn’t have all the functionality we want it to have. In some development environments, it fails by displaying a red bar—thus the red in the mnemonic.

In the second step, making it pass, we implement the missing functionality so that all tests pass—both the one we just added and all the ones we had already.

At this time, the red bar turns to green, which takes us to green in the mnemonic. The last part of the cycle, refactor, is just that—refactoring. As we improve the design of the code without altering its external behavior, all tests should pass and, thus, we should remain green.

Red, green, green. Red, green, refactor. Quite catchy, isn’t it?

In its deceptive simplicity, this little cycle, test-code-refactor, encompasses a significant power to improve the overall quality of our personal software process and, subsequently, that of the whole team, project, and organization.

Reference: Test Driven PRACTICAL TDD AND ACCEPTANCE TDD FOR JAVA DEVELOPERS by LASSE KOSKELA - Manning 2008

Management Skills - Networking

2009-10-23T03:48:00.000-07:00

Increasingly in today’s more fluid and flexible organizations, people get things done by networking.
Networks are loosely organized connections between people with shared interests. Networking takes place within them when people exchange information, enlist support and create alliances getting agreement with other people on a course of action and joining forces to make it happen.

It occurs outside the usual formal communication channels. It is an essential way of getting things done in organizations – it ensures that the informal organization works.

Networks inside organizations are often fluid and informal. They exist to meet a need and can be dispersed if that need no longer exists, only to be reformed when it reappears. Networks may just consist of people with similar aims or interests who communicate with one another or get together as required.

Networks are sometimes set up formally in organizations, for example the ‘communities of interest’ that are created to exchange and share knowledge and experience as part of a ‘knowledge management’ programme.

Networks can also exist outside the organization. Again, they may consist of like-minded individuals exchanging information and meeting informally, or they may be set up formally with regular meetings and newsletters.

Here are 10 steps you can take to network effectively:

1. Identify people who may be able to help.

2. Seize any opportunity that presents itself to get to know people who may be useful.

3. Have a clear idea of why you want to network – to share knowledge, to persuade people to accept your proposal or point of view, or to form an alliance.

4. Know what you can contribute – networking is not simply about enlisting support, it is just as much if not more concerned with developing knowledge and understanding and joining forces with like-minded people so that concerted effort can be deployed to get things done.

5. Show interest – if you engage with people and listen to them, they are more likely to want to network with you.

6. Ask people if you can help them as well as asking people to help you.

7. Put people in touch with one another.

8. Operate informally but be prepared to call formal meetings when necessary to reach agreement and plan action.

9. Make an effort to keep in touch with people.

10. Follow up – check with members of the network on progress in achieving something, refer back to conversations you have had, discuss with others how the network might be developed or extended to increase its effectiveness.

Management Skills - Problem Solving

2009-10-23T03:43:00.000-07:00

Peter Drucker (1967) wrote that ‘In every area of effectiveness within an organization, one feeds the opportunity and starves the problem’. It is indeed often said that ‘there are no problems, only opportunities’.

This is not universally true of course, but it does emphasize the point
that a problem should lead to positive thinking about what is to be done now. It is not the time for recriminations. If a mistake has been made, the reasons for it should be analysed so that it does not happen again.

The following are 10 steps for effective problem-solving:

1. Define the situation – establish what has gone wrong or is potentially going to go wrong.

2. Specify objectives – define what is to be achieved now or in the future to deal with an actual or potential problem or a change in circumstances.

3. Develop hypotheses – come up with theories about what has caused the problem.

4. Get the facts – find out what has actually happened and contrast this with an assessment of what ought to have happened. Obtain information about internal or external constraints that affect the situation. However, remember what Nietzsche (1883) wrote: ‘There are no facts, only interpretations’. Try to understand the attitudes and motivation of those concerned. Remember that people will see what has happened in terms of their own position and feelings (their framework of reference).

5. Analyse the facts – determine what is relevant and what is irrelevant. Diagnose the likely cause or causes of the problem. Do not be tempted to focus on symptoms rather than root causes. Test any assumptions. Dig into what lies behind the problem.

6. Identify possible courses of action – spell out what each involves.

7. Evaluate alternative courses of action – assess the extent to which they are likely to achieve the objectives, the cost of implementation, any practical difficulties that might emerge and the possible reactions of stakeholders.

8. Weigh and decide – determine which alternative is likely to result in the most practical and acceptable solution to the problem. This is often a balanced judgement.

9. Plan implementation – timetable, project management, and resources required.

10. Implement – monitor progress and evaluate success. Remember that a problem has not been solved until the decision has been implemented. Always work out the solution to a problem with implementation in mind.

Management Skills - Decision Making

2009-10-23T03:37:00.000-07:00

Good managers are decisive. They can quickly size up a situation and reach the right conclusion about what should be done about it. To say of someone ‘He or she is decisive’ is praise indeed, as long as the decisions are effective.

Characteristics of the decision-making process

Decision-making is about analysing the situation or problem, identifying possible courses of action, weighing them up and defi ning a course of action.

Peter Drucker (1967) says: ‘A decision is a judgement. It is a choice between alternatives. It is rarely a choice between right and wrong. It is at best a choice between almost right and probably wrong – but much more often a choice between two courses of action neither of which is probably more nearly right than the other.’

You should not expect or even welcome a bland consensus view. The best decisions emerge from conflicting viewpoints. This is Drucker’s first law of decision-making: ‘One does not make a decision without disagreements’.

You can benefit from a clash of opinion as it prevents people falling
into the trap of starting with the conclusion and then looking for the facts that support it.

Alfred P Sloan of General Motors knew this. At a meeting of one of his top committees he said:

‘Gentlemen, I take it we are all in agreement on the decision here.’ Everyone around the table nodded assent. ‘Then,’ continued Mr Sloan, ‘I propose we postpone further discussion of the matter until our next meeting to give ourselves time to develop disagreement and perhaps gain
some understanding of what the decision is all about.’

Ten approaches to being decisive

1. Make decisions faster – Jack Welch, when heading General Electric, used to say: ‘In today’s lightning-paced environment, you don’t have time to think about things. Don’t sit on decisions. Empty that in-basket so that you are free to search out new opportunities…
Don’t sit still. Anybody sitting still, you are going to guarantee they’re going to get their legs knocked from under them.’

2. Avoid procrastination – it is easy to put an e-mail demanding a decision into the ‘too difficult’ section of your actual or mental in-tray. Avoid the temptation to fill your time with trivial tasks so that the evil moment when you have to address the issue is postponed.

Make a start. Once you have got going you can deal with the unpleasant task of making a decision in stages. A challenge often becomes easier once we have started dealing with it.
Having spent five minutes on it, we don’t want to feel it was wasted, so we carry on and complete the job.

3. Expect the unexpected – you are then in the frame of mind needed to respond decisively to a new situation.

4. Think before you act – this could be a recipe for delay, but decisive people use their analytical ability to come to swift conclusions about the nature of the situation and what should be done about it.

5. Be careful about assumptions – we have a tendency to leap to conclusions and seize on assumptions that support our case, ignoring the facts that might contradict it.

6. Learn from the past – build on your experience in decision-making; what approaches work best. But don’t rely too much on precedents. Situations change. The right decision last time could well be the wrong one now.

7. Be systematic – adopt a rigorous problem-solving approach, This means specifying objectives (what you want to achieve), defining the criteria for judging whether they have been achieved, getting and analysing the facts, looking for causes rather than focusing on symptoms, developing and testing hypotheses and alternative solutions, and evaluating possible causes of action against the objectives and criteria.

8. Talk it through – before you make a signifi cant decision talk it through with someone who is likely to disagree so that any challenge they make can be taken into account (but you have to canvass opinion swiftly).

9. Leave time to think it over – swift decision-making is highly desirable but you must avoid knee-jerk reactions. Pause, if only for a few minutes, to allow yourself time to think through the decision you propose to make. Confirm that it is logical and fully justified.

10. Consider the potential consequences – McKinsey, the management consultants, call this ‘consequence management’. Every decision has a consequence and you should consider very carefully what that might be and how you would manage it. When making a decision it is a good idea to start from where you mean to end – define the end result and then work out the steps needed to achieve it.

Management Skills - Motivating

2009-10-23T03:30:00.000-07:00

Leadership is about getting people into action and ensuring that
they continue taking that action in order to complete the task. It is therefore very much about motivation. This can be defined as the process of getting people to move in the direction the leader wants them to go.

The organization as a whole provides the context within which high
levels of motivation can be achieved, through reward systems and the provision of opportunities for growth and development. But managers still have a major part to play in deploying their motivating skills to ensure that people give of their best.

The aim is to get people to exert the maximum amount of positive discretionary effort – they often have a choice about how they carry out their work and the amount of care, innovation and productive behaviour they
display. Discretionary effort makes the difference between people just doing a job and people doing a great job.

What is motivation?

A motive is a reason for doing something. Motivation is concerned with the factors that influence people to behave in certain ways. Motivating other people is about getting them to move in the direction you want them to go in order to achieve a result. Motivation can be described as goal-directed behaviour.

Motivation is initiated by the conscious or unconscious recognition of an unsatisfied need. A goal is then established that it is believed will satisfy this need, and a decision is made on the action that it is expected will achieve the goal. If the goal is achieved the need will be satisfi ed and the behaviour is likely to be repeated the next time a similar need emerges. If the goal is not
achieved the same action is less likely to be repeated.

Ten ways of motivating people

1. Agree demanding but achievable goals.

2. Create expectations that certain behaviours and outputs will produce worthwhile rewards when people succeed.

3. Provide feedback on performance.

4. Design jobs that enable people to feel a sense of accomplishment, to express and use their abilities, and to exercise their own decision-making powers.

5. Make good use of the organization’s reward system to provide appropriate financial incentives.

6. Provide recognition and praise for work well done.

7. Communicate to your team and its members the link between performance and reward, thus enhancing expectations.

8. Provide effective leadership.

9. Give people the guidance and training that will develop the knowledge and skills they need to improve their performance and be rewarded accordingly.

10. Offer opportunities for learning and development that will enable them to advance their careers.

Strategic Management - Scenario Planning

2009-10-14T03:48:00.000-07:00

Traditional forecasting techniques often fail to predict significant changes in the firm's external environment, especially when the change is rapid and turbulent or when information is limited. Consequently, important opportunities and serious threats may be overlooked and the very survival of the firm may be at stake.

Scenario planning is a tool specifically designed to deal with major, uncertain shifts in the firm's environment.
Scenario planning has its roots in military strategy studies. Herman Kahn was an early founder of scenario-based planning in his work related to the possible scenarios associated with thermonuclear war ("thinking the unthinkable").

Scenario planning was transformed into a business tool in the late 1960's and early 1970's, most notably by Pierre Wack who developed the scenario planning system used by Royal Dutch/Shell. As a result of these efforts, Shell was prepared to deal with the oil shock that occurred in late 1973 and greatly improved its competitive position in the industry during the oil crisis and the oil glut that followed.

Scenario planning is not about predicting the future. Rather, it attempts to describe what is possible. The result of a scenario analysis is a group of distinct futures, all of which are plausible. The challenge then is how to deal with each of the possible scenarios.

Scenario planning often takes place in a workshop setting of high level executives, technical experts, and industry leaders. The idea is to bring together a wide range of perspectives in order to consider scenarios other than the widely accepted forecasts. The scenario development process should include interviews with managers who later will formulate and implement strategies based on the scenario analysis - without their input the scenarios may leave out important details and not lead to action if they do not address issues important to those who will implement the strategy.

Some of the benefits of scenario planning include:

Managers are forced to break out of their standard world view, exposing blind spots that might otherwise be overlooked in the generally accepted forecast.
Decision-makers are better able to recognize a scenario in its early stages, should it actually be the one that unfolds.
Managers are better able to understand the source of disagreements that often occur when they are envisioning different scenarios without realizing it.

The Scenario Planning Process

The following outlines the sequence of actions that may constitute the process of scenario planning.

Specify the scope of the planning and its time frame.
For the present situation, develop a clear understanding that will serve as the common departure point for each of the scenarios.
Identify predetermined elements that are virtually certain to occur and that will be driving forces.
Identify the critical uncertainties in the environmental variables. If the scope of the analysis is wide, these may be in the macro-environment, for example, political, economic, social, and technological factors (as in PEST).
Identify the more important drivers. One technique for doing so is as follows. Assign each environmental variable two numerical ratings: one rating for its range of variation and another for the strength of its impact on the firm. Multiply these ratings together to arrive at a number that specifies the significance of each environmental factor. For example, consider the extreme case in which a variable had a very large range such that it might be rated a 10 on a scale of 1 to 10 for variation, but in which the variable had very little impact on the firm so that the strength of impact rating would be a 1. Multiplying the two together would yield 10 out of a possible 100, revealing that the variable is not highly critical. After performing this calculation for all of the variables, identify the two having the highest significance.
Consider a few possible values for each variable, ranging between extremes while avoiding highly improbable values.
To analyze the interaction between the variables, develop a matrix of scenarios using the two most important variables and their possible values. Each cell in the matrix then represents a single scenario. For easy reference in later discussion it is worthwhile to give each scenario a descriptive name. If there are more than two critical factors, a multidimensional matrix can be created to handle them but would be difficult to visualize beyond 2 or 3 dimensions. Alternatively, factors can be taken in pairs to generate several two-dimensional matrices. A scenario matrix might look something like this:

Scenario Matrix

VARIABLE 1

Outcome 1A
|
V

Outcome 1B
|
V

V
A
R
I
A
B
L
E

2

Outcome 2A -->
Scenario 1 Scenario 2

Outcome 2B -->
Scenario 3 Scenario 4

One of these scenarios most likely will reflect the mainstream views of the future. The other scenarios will shed light on what else is possible.
At this point there is not any detail associated with these "first-generation" scenarios. They are simply high level descriptions of a combination of important environmental variables. Specifics can be generated by writing a story to develop each scenario starting from the present. The story should be internally consistent for the selected scenario so that it describes that particular future as realistically as possible. Experts in specific fields may be called upon to devlop each story, possibly with the use of computer simulation models. Game theory may be used to gain an understanding of how each actor pursuing its own self interest might respond in the scenario. The goal of the stories is to transform the analysis from a simple matrix of the obvious range of environmental factors into decision scenarios useful for strategic planning.
Quantify the impact of each scenario on the firm, and formulate appropriate strategies.

An additional step might be to assign a probability to each scenario. Opinions differ on whether one should attempt to assign probabilities when there may be little basis for determining them.

Business unit managers may not take scenarios seriously if they deviate too much from their preconceived view of the world. Many will prefer to rely on forecasts and their judgement, even if they realize that they may miss important changes in the firm's environment. To overcome this reluctance to broaden their thinking, it is useful to create "phantom" scenarios that show the adverse results if the firm were to base its decisions on the mainstream view while the reality turned out to be one of the other scenarios.

Linear Programming

2009-10-14T03:45:00.000-07:00

Operations management often presents complex problems that can be modeled by linear functions. The mathematical technique of linear programming is instrumental in solving a wide range of operations management problems.

Linear Program Structure

Linear programming models consist of an objective function and the constraints on that function. A linear programming model takes the following form:

Objective function:

Z = a₁X₁ + a₂X₂ + a₃X₃ + . . . + a_nX_n

Constraints:
      b₁₁X₁ + b₁₂X₂ + b₁₃X₃ + . . . + b_1nX_n < c₁
      b₂₁X₁ + b₂₂X₂ + b₂₃X₃ + . . . + b_2nX_n < c₂
      .
      .
      .
      b_m1X₁ + b_m2X₂ + b_m3X₃ + . . . + b_mnX_n < c_m

In this system of linear equations, Z is the objective function value that is being optimized, X_i are the decision variables whose optimal values are to be found, and a_i, b_ij, and c_i are constants derived from the specifics of the problem.

Linear Programming Assumptions

Linear programming requires linearity in the equations as shown in the above structure. In a linear equation, each decision variable is multiplied by a constant coefficient with no multiplying between decision variables and no nonlinear functions such as logarithms. Linearity requires the following assumptions:

Proportionality - a change in a variable results in a proportionate change in that variable's contribution to the value of the function.
Additivity - the function value is the sum of the contributions of each term.
Divisibility - the decision variables can be divided into non-integer values, taking on fractional values. Integer programming techniques can be used if the divisibility assumption does not hold.

In addition to these linearity assumptions, linear programming assumes certainty; that is, that the coefficients are known and constant.

Problem Formulation

With computers able to solve linear programming problems with ease, the challenge is in problem formulation - translating the problem statement into a system of linear equations to be solved by computer. The information required to write the objective function is derived from the problem statement. The problem is formulated from the problem statement as follows:

Identify the objective of the problem; that is, which quantity is to be optimized. For example, one may seek to maximize profit.
Identify the decision variables and the constraints on them. For example, production quantities and production limits may serve as decision variables and constraints.
Write the objective function and constraints in terms of the decision variables, using information from the problem statement to determine the proper coefficient for each term. Discard any unnecessary information.
Add any implicit constraints, such as non-negative restrictions.
Arrange the system of equations in a consistent form suitable for solving by computer. For example, place all variables on the left side of their equations and list them in the order of their subscripts.

The following guidelines help to reduce the risk of errors in problem formulation:

Be sure to consider any initial conditions.
Make sure that each variable in the objective function appears at least once in the constraints.
Consider constraints that might not be specified explicitly. For example, if there are physical quantities that must be non-negative, then these constraints must be included in the formulation.

The Effect of Constraints

Constraints exist because certain limitations restrict the range of a variable's possible values. A constraint is considered to be binding if changing it also changes the optimal solution. Less severe constraints that do not affect the optimal solution are non-binding.

Tightening a binding constraint can only worsen the objective function value, and loosening a binding constraint can only improve the objective function value. As such, once an optimal solution is found, managers can seek to improve that solution by finding ways to relax binding constraints.

Shadow Price

The shadow price for a constraint is the amount that the objective function value changes per unit change in the constraint. Since constraints often are determined by resources, a comparison of the shadow prices of each constraint provides valuable insight into the most effective place to apply additional resources in order to achieve the best improvement in the objective function value.
The reported shadow price is valid up to the allowable increase or allowable decrease in the constraint.

Applications of Linear Programming

Linear programming is used to solve problems in many aspects of business administration including:

product mix planning
distribution networks
truck routing
staff scheduling
financial portfolios
corporate restructuring

The Marketing Process

2009-10-14T03:43:00.000-07:00

Under the marketing concept, the firm must find a way to discover unfulfilled customer needs and bring to market products that satisfy those needs. The process of doing so can be modeled in a sequence of steps: the situation is analyzed to identify opportunities, the strategy is formulated for a value proposition, tactical decisions are made, the plan is implemented and the results are monitored.

The Marketing Process

Situation Analysis

|
V

Marketing Strategy

|
V

Marketing Mix Decisions

|
V

Implementation & Control

I. Situation Analysis

A thorough analysis of the situation in which the firm finds itself serves as the basis for identifying opportunities to satisfy unfulfilled customer needs. In addition to identifying the customer needs, the firm must understand its own capabilities and the environment in which it is operating.
The situation analysis thus can be viewed in terms an analysis of the external environment and an internal analysis of the firm itself. The external environment can be described in terms of macro-environmental factors that broadly affect many firms, and micro-environmental factors closely related to the specific situation of the firm.

The situation analysis should include past, present, and future aspects. It should include a history outlining how the situation evolved to its present state, and an analysis of trends in order to forecast where it is going. Good forecasting can reduce the chance of spending a year bringing a product to market only to find that the need no longer exists.

If the situation analysis reveals gaps between what consumers want and what currently is offered to them, then there may be opportunities to introduce products to better satisfy those consumers. Hence, the situation analysis should yield a summary of problems and opportunities. From this summary, the firm can match its own capabilities with the opportunities in order to satisfy customer needs better than the competition.
There are several frameworks that can be used to add structure to the situation analysis:

5 C Analysis - company, customers, competitors, collaborators, climate. Company represents the internal situation; the other four cover aspects of the external situation
PEST analysis - for macro-environmental political, economic, societal, and technological factors. A PEST analysis can be used as the "climate" portion of the 5 C framework.
SWOT analysis - strengths, weaknesses, opportunities, and threats - for the internal and external situation. A SWOT analysis can be used to condense the situation analysis into a listing of the most relevant problems and opportunities and to assess how well the firm is equipped to deal with them.

II. Marketing Strategy

Once the best opportunity to satisfy unfulfilled customer needs is identified, a strategic plan for pursuing the opportunity can be developed. Market research will provide specific market information that will permit the firm to select the target market segment and optimally position the offering within that segment. The result is a value proposition to the target market. The marketing strategy then involves:

Segmentation
Targeting (target market selection)
Positioning the product within the target market
Value proposition to the target market

III. Marketing Mix Decisions

Detailed tactical decisions then are made for the controllable parameters of the marketing mix. The action items include:

Product development - specifying, designing, and producing the first units of the product.
Pricing decisions
Distribution contracts
Promotional campaign development

IV. Implementation and Control

At this point in the process, the marketing plan has been developed and the product has been launched. Given that few environments are static, the results of the marketing effort should be monitored closely. As the market changes, the marketing mix can be adjusted to accomodate the changes. Often, small changes in consumer wants can addressed by changing the advertising message. As the changes become more significant, a product redesign or an entirely new product may be needed. The marketing process does not end with implementation - continual monitoring and adaptation is needed to fulfill customer needs consistently over the long-term.

Economics - Supply and Demand

2009-10-14T03:40:00.000-07:00

The market price of a good is determined by both the supply and demand for it. In 1890, English economist Alfred Marshall published his work, Principles of Economics, which was one of the earlier writings on how both supply and demand interacted to determine price. Today, the supply-demand model is one of the fundamental concepts of economics. The price level of a good essentially is determined by the point at which quantity supplied equals quantity demanded. To illustrate, consider the following case in which the supply and demand curves are plotted on the same graph.

Supply and Demand

On this graph, there is only one price level at which quantity demanded is in balance with the quantity supplied, and that price is the point at which the supply and demand curves cross.

The law of supply and demand predicts that the price level will move toward the point that equalizes quantities supplied and demanded. To understand why this must be the equilibrium point, consider the situation in which the price is higher than the price at which the curves cross. In such a case, the quantity supplied would be greater than the quantity demanded and there would be a surplus of the good on the market. Specifically, from the graph we see that if the unit price is $3 (assuming relative pricing in dollars), the quantities supplied and demanded would be:

Quantity Supplied = 42 units

Quantity Demanded = 26 units

Therefore there would be a surplus of 42 - 26 = 16 units. The sellers then would lower their price in order to sell the surplus.

Suppose the sellers lowered their prices below the equilibrium point. In this case, the quantity demanded would increase beyond what was supplied, and there would be a shortage. If the price is held at $2, the quantity supplied then would be:

Quantity Supplied = 28 units

Quantity Demanded = 38 units

Therefore, there would be a shortage of 38 - 28 = 10 units. The sellers then would increase their prices to earn more money.

The equilibrium point must be the point at which quantity supplied and quantity demanded are in balance, which is where the supply and demand curves cross. From the graph above, one sees that this is at a price of approximately $2.40 and a quantity of 34 units.

To understand how the law of supply and demand functions when there is a shift in demand, consider the case in which there is a shift in demand:

Shift in Demand

In this example, the positive shift in demand results in a new supply-demand equilibrium point that in higher in both quantity and price. For each possible shift in the supply or demand curve, a similar graph can be constructed showing the effect on equilibrium price and quantity. The following table summarizes the results that would occur from shifts in supply, demand, and combinations of the two.

Result of Shifts in Supply and Demand

Demand	Supply	Equilibrium Price	Equilibrium Quantity
+		+	+
-		-	-
	+	-	+
	-	+	-
+	+	?	+
-	-	?	-
+	-	+	?
-	+	-	?

In the above table, "+" represents an increase, "-" represents a decrease, a blank represents no change, and a question mark indicates that the net change cannot be determined without knowing the magnitude of the shift in supply and demand. If these results are not immediately obvious, drawing a graph for each will facilitate the analysis.

Herzberg's Motivation-Hygiene Theory

2009-10-14T03:37:00.001-07:00

To better understand employee attitudes and motivation, Frederick Herzberg performed studies to determine which factors in an employee's work environment caused satisfaction or dissatisfaction. He published his findings in the 1959 book The Motivation to Work.

The studies included interviews in which employees where asked what pleased and displeased them about their work. Herzberg found that the factors causing job satisfaction (and presumably motivation) were different from those causing job dissatisfaction. He developed the motivation-hygiene theory to explain these results. He called the satisfiers motivators and the dissatisfiers hygiene factors, using the term "hygiene" in the sense that they are considered maintenance factors that are necessary to avoid dissatisfaction but that by themselves do not provide satisfaction.

The following table presents the top six factors causing dissatisfaction and the top six factors causing satisfaction, listed in the order of higher to lower importance.

Factors Affecting Job Attitudes

Leading to Dissatisfaction	Leading to Satisfaction
Company policy Supervision Relationship w/Boss Work conditions Salary Relationship w/Peers	Achievement Recognition Work itself Responsibility Advancement Growth

Herzberg reasoned that because the factors causing satisfaction are different from those causing dissatisfaction, the two feelings cannot simply be treated as opposites of one another. The opposite of satisfaction is not dissatisfaction, but rather, no satisfaction. Similarly, the opposite of dissatisfaction is no dissatisfaction.

While at first glance this distinction between the two opposites may sound like a play on words, Herzberg argued that there are two distinct human needs portrayed. First, there are physiological needs that can be fulfilled by money, for example, to purchase food and shelter. Second, there is the psychological need to achieve and grow, and this need is fulfilled by activities that cause one to grow.

From the above table of results, one observes that the factors that determine whether there is dissatisfaction or no dissatisfaction are not part of the work itself, but rather, are external factors. Herzberg often referred to these hygiene factors as "KITA" factors, where KITA is an acronym for Kick In The A..., the process of providing incentives or a threat of punishment to cause someone to do something. Herzberg argues that these provide only short-run success because the motivator factors that determine whether there is satisfaction or no satisfaction are intrinsic to the job itself, and do not result from carrot and stick incentives.

Implications for Management

If the motivation-hygiene theory holds, management not only must provide hygiene factors to avoid employee dissatisfaction, but also must provide factors intrinsic to the work itself in order for employees to be satisfied with their jobs.

Herzberg argued that job enrichment is required for intrinsic motivation, and that it is a continuous management process. According to Herzberg:

The job should have sufficient challenge to utilize the full ability of the employee.
Employees who demonstrate increasing levels of ability should be given increasing levels of responsibility.
If a job cannot be designed to use an employee's full abilities, then the firm should consider automating the task or replacing the employee with one who has a lower level of skill. If a person cannot be fully utilized, then there will be a motivation problem.

Critics of Herzberg's theory argue that the two-factor result is observed because it is natural for people to take credit for satisfaction and to blame dissatisfaction on external factors. Furthermore, job satisfaction does not necessarily imply a high level of motivation or productivity.

Herzberg's theory has been broadly read and despite its weaknesses its enduring value is that it recognizes that true motivation comes from within a person and not from KITA factors.

Revolution of Semantic Web Services

2009-10-03T01:21:00.001-07:00

Web services and service oriented architectures were presented as a significant step towards a simple solution for the interoperability problems that typically plague middleware applications.

Web services also play a part in the Semantic Web. As Semantic Web applications increase in complexity, and as information consumers become more discerning, the focus is turning from semantically addressable information to semantically addressable services that allow automated creation of customized software system, or Semantic Web services.

Current tools provide the capability to describe Web services, but do not have adequate means for categorizing and utilizing those descriptions. The categorizations available, such as WSIndex are designed primarily for human use rather than machine.

As a result, automated system composition is more a dream than reality right now.

However, unlike most information on the Web, Web services are typically accompanied by adequate metadata, which is the key component of the Semantic Web. The difficulty in using that metadata lies in understanding the semantics. Predictably, there is a substantial amount of research focused

on applying ontologies and other tools of the Semantic Web to Web services.

Semantic Web services provide a mechanism for creating semantically rich descriptions of services. These can be used to create specifications for composite services, to represent business logic at a more abstract level and to supply knowledge for reasoning systems which can then intelligently assemble software from service descriptions.

One of the underlying languages in the development of Semantic Web services is the Ontology

Web Language for Services (OWL-S). OWL-S is an ontology language with the goal of enabling automated discovery, incorporation and execution of Web services.

OWL-S breaks the description of a service into three parts, a service profile, a process model and a mapping between the process model and the message passing protocol (called a grounding).

The profile is used both to advertise services and to request services. The intention is that the profile

is used in service discovery to match appropriate services to requests for services. Profiles consist, in part, of a description of both functional parameters, described as inputs, outputs, preconditions and effects, and nonfunctional parameters such text fields meant for human consumption or to

define quality of service constraints.

The process model uses the OWL-S process ontology to describe the execution of a web service. The process model precisely describes the required flow of control and data to obtain the results specified in the profile and includes three types of processes, namely atomic, simple and composite.

Only atomic processes may be executed directly.

Finally the grounding provides the binding between the atomic processes and the message format selected. Currently OWL-S has a grounding ontology defined for WDSL that allows for describing the message requirements for atomic processes.

OWL-S is not a complete specification nor is it a stable standard. It is,

however, a good start at defining how semantics can be added to Web services. It has been subjected to few real world test cases to date, but experiences with a prototypical use case implementation suggested that OWL-S may be a good start for enterprise integration, but that it currently lacks the capacity to allow fully automated discovery and composition of services.

Nonetheless, it seems inevitable that OWL-S or some other similar work, such as the Internet Reasoning Service will result in an automated mechanism for finding and composing services.

The CRAM HR Management Model

2009-09-25T09:27:00.000-07:00

Never lose sight of the fact that the members of your project team are
human beings, with aspirations, strengths, constraints, and weaknesses. Your
project’s success hinges more on team members’ attitudes and aptitudes than it does on your Gantt chart wizardry and project tracking prowess. Feel free to manage the project, but don’t forget to lead the team.

Many of us manage projects in a matrix environment with team members
reporting both to us and to a department manager. We do not have human
resources (HR) hiring/firing/evaluation responsibility for them. However,
don’t abdicate responsibility for the care and feeding of the people on the team to managers in the HR or functional hierarchy.

Many of those managers get promoted based on technical knowledge of
human resources or their departments, not on their ability to inspire people.
Your project’s success depends on your ability to lead. There are many books
available on leadership. Read voraciously.

Everyone on your team wants to contribute, learn, and achieve. It may be challenging at times to dig deeply enough to find this desire in some team members, but it’s what makes software project management challenging and fun.

Hold one-on-one conversations with your team members regularly. Determine
what their issues are, ask them for ideas, and give them a voice in the
project. Take their input seriously and act on it.

Ask your team members what they want to be when they grow up. Seriously.
We all have career aspirations. Be the one mentor who cares about their
careers. You’ll be amazed at how powerful this can be.

Be open, honest, and direct with team members. Provide feedback on a regular
basis, not just at review time. Focus your feedback on the behavior, not the
person. Again, management literature abounds. Study.

When you have a performance issue with a team member, apply the CRAM
model: Constraints, Resources, Aptitude, and Motivation.

Project managers frequently diagnose poor performance as a motivation problem.
The CRAM model suggests that motivation is the last issue to consider. A team member may be experiencing constraints in his life that limit his effectiveness.

Examples include getting divorced or married, having kids, fighting addiction issues, etc.
Team members may not have the resources necessary to contribute at their
highest level. Examples include no quality assurance (QA) test environment,
or ancient hardware. Perhaps budget constraints limit the ability to establish
testing environments or buy licenses for necessary software.

Perhaps the domain expertise (business analyst, customer, end-user) is not accessible. Your team member may not be cut out for the role he/she fills. He may not have the programming aptitude necessary for this project. If so, find another project role, if possible. Alternatively, find another team where he can leverage his strengths.

Motivation is the last lever to jiggle when a team member has performance
issues. It should only be considered once the constraints, resources, and aptitude problems have been addressed.

Be a leader and connect with the individual human beings who comprise your
team. The results may surprise you.

Introduce a More Agile Communication System

2009-09-25T09:18:00.000-07:00

Most retrospectives of failed projects place a great deal of blame
on communication breakdowns between software project managers, team
members, and stakeholders.

Project managers are taught to mitigate communication breakdowns between team members, and provide constant, effective communication. The weight of this responsibility sometimes leads project managers to overreact. They blur the line between essential, concrete communications and those where the content‐to‐noise ratio actually harms project progress instead of helping it.

To solve this problem, many software development endeavors are moving
toward a more flexible, agile process. The key to agile methodologies is timely
communication loops that enable agile teams to respond effectively to unforeseen changes, and quickly reassess and reprioritize project features.

How do agile project managers keep communications limited to the essentials?

They promote the daily “15-minute stand‐up” meeting. It entails developers
describing what they’ve accomplished since the last standup, what
they’re planning to accomplish “today,” and any impediments they foresee in
reaching their goals. The negative risk of a stand‐up meeting is that it may
rely solely on the precision of each developer’s self-assessment. The solution?

To make stand‐up meetings more effective, integrate a task management tool
that can show the output of a feature’s tests. A tool does not lie about the state of a project’s codebase, and testing results are a valuable addition to developer self-assessment. Presenting report data correlating a feature to a set of tests it passed gives a more accurate representation of the state of the feature.

For example, results from a continuous integration tool can paint an objective
picture of progress. This reduces the stand‐up meeting communication to
the essentials: reporting of impediments (hopefully, already caught by the task
management tool) and unforeseen developments due to edge cases, integration challenges, and bugs/defects. By reflecting these development “discoveries” through a shared, globally accessible tool, developers gain a greater level of feedback precision. Often, unseen connections between features and tasks can be discovered early.

One typical misconception is that synchronous communications are always
more effective than asynchronous† communications. Adding development
tools and short, asynchronous communication loops can effectively supplement face‐to‐face communications.

At a more general level of feedback, a wiki system can easily keep the vision
of the project adjusted to the reality of the development progress. Such a system can also make information available in a timely manner and provide a
higher‐level channel to communicate to stakeholders-at-large, who might not
be interested in the deep, technical details impeding a particular feature’s progress.

By contrast, a software developer’s vision of the overall project can be
blurred over time by the minutiae of his daily technical work. A wiki is an effective way to keep a clear, shared vision of the project among all the participants.

By attacking the problem of keeping information loops tight and noise-free,
software project managers can help avoid the breakdown in communications
typically blamed for project failures. A project manager’s responsibility and
challenge is to streamline the feedback loops at every level of a project.

The 60/60 Rule

2009-09-25T07:11:00.000-07:00

We often pretend that software development is the most important part
of the software life cycle. Methodologies abound for development. Books,
magazine articles, and blogs focus on development. Development, however, is
just not where the money is.

Fully 60% of the life cycle costs of software systems come from maintenance,
with a relatively measly 40% coming from development. That is an average, of
course.

The actual cost of maintenance may vary from 40% to 80%, depending
on the system type and the environment it is deployed into. During maintenance, 60% of the costs on average relate to user-generated enhancements (changing requirements), 23% to migration activities, and 17% to bug fixes. The 60% of life cycle costs related to maintenance, coupled with the fact that 60% of maintenance activities relate to enhancements, gives us the so-called 60/60 Rule, one of the few proposed “laws” of software maintenance.

Migration activities include moving systems to new hardware or software environments.

Migration is, of course, a type of changing requirement. Factoring
that into our estimates points out an interesting fact: over 80% of maintenance activities relate in some way to changing requirements.

Naturally, the ability to change code suggests that one should understand it
first. Understanding changes to be made is a major activity during maintenance.

Roughly 30% of total maintenance time is spent on understanding an
existing software product. The development of understanding applies to all
forms of maintenance: changing requirements, migration, and bug fixes.

Understanding is a cost we must pay to maintain code that someone else wrote, or we wrote long enough ago that we no longer have an intimate knowledge of it. During maintenance, understanding code takes the place of new design work found during development for most tasks.

The 60/60 Rule should prompt us to rethink the focus of software development,
as well as maintenance. The tendency to address development activities
may not yield the most impressive gains.

Boehm’s early assertion in the early 1980s that proper software engineering discipline can reduce defects by up to 75% may be true (although I seriously doubt it), and became the basis for much work toward development methodologies, but so what?

A good methodology may reduce bugs (17% of the total maintenance effort), but not address migration, enhancement time, or cost at all. To reduce maintenance costs, we have to address the costs associated with understanding code, adjusting code to new requirements, and/or migrating code to new environments.

The 60/60 Rule suggests that we should focus our efforts on creating systems that are maintainable. Our software must be designed to change so systems become flexible in the face of new requirements. Designing such systems is one of the next great challenges in software engineering.

We know at least part of the answer. The software components need to become less tightly coupled with one another, much the way the components of the World Wide Web are bound together at the last possible moment and in a flexible manner.

		VARIABLE 1
		Outcome 1A \| V	Outcome 1B \| V
V A R I A B L E 2	Outcome 2A -->	Scenario 1	Scenario 2
V A R I A B L E 2	Outcome 2B -->	Scenario 3	Scenario 4

MyOpenDraft

The challenges for Informatics in developing software for modern multikernel computers

The Management 3.0 - Leading Agile Developers, Developing Agile Leaders

Six Thinking Hats

Packt GlassFish Security book contest

Best Practices for building Large Scale Agile teams

Role of UML in Agile & Evolutionary Architecture

Best Practices of Agile Distributed Teams

All About User Stories

Agile for Today Competitive Business Environment

Lean Thinking

Introduction to Behaviour-Driven Development with Groovy

Challenges of Scaling Agile in Enterprises

J2EE 6 - Bean Validation

J2EE 6 - Contexts and Dependency Injection

J2EE 6 - Java API for RESTful Web Services (JAX-RS)

Introduction to Database Marketing

Scrum - The Rugby Strategy in Product Development

22 Lean Software Development tools

Agile Simplified with Feature Driven Development

The 12 Practices of Extreme Programming

Nine Principles of Dynamic Systems Development Method

Introduction to Adaptive Software Development

Motivation and Self Organization teams from Agile Perspective

People over processes and tools Or Balancing people, process, and tools

The 12 Principles of Agile Methods

Lean Versus Agile

Lean Software Development

The Lean Success Story

New Features and Enhancements in Spring 3.0

What's New in Java EE 6

Pruning

What is the biggest major criticism of Java EE? Its simply too large. The fact is, majority of Java web applications do not utilize/leverage the full Java EE stack. So why is that stuff still in there?

Java EE 6 has arrived!

Envisioning Next Generation of Web - The Semantic Web

Learning Organization in Practice - Senge’s View "The Fifth Discipline"

What is The Open Source Business Model?

Another Perspective – Tests as a shared language

Software Development with Close collaboration

Keeping code healthy with refactoring

Only ever write code to fix a failing test

Management Skills - Networking

Management Skills - Problem Solving

Management Skills - Decision Making

Management Skills - Motivating

Strategic Management - Scenario Planning

The Scenario Planning Process

Scenario Matrix

Linear Programming

Linear Program Structure

Linear Programming Assumptions

Problem Formulation

The Effect of Constraints

Shadow Price

Applications of Linear Programming

The Marketing Process

The Marketing Process

I. Situation Analysis

II. Marketing Strategy

III. Marketing Mix Decisions

IV. Implementation and Control

Economics - Supply and Demand

Supply and Demand

Shift in Demand

Result of Shifts in Supply and Demand

Herzberg's Motivation-Hygiene Theory

Factors Affecting Job Attitudes

Implications for Management

Revolution of Semantic Web Services

The CRAM HR Management Model

Introduce a More Agile Communication System

The 60/60 Rule