Something Big is Going to Happen ... Iran Agile 2013

"...everything is changing very fast and unpredictably. The market requires low volume, high quality, custom and specific products… These products have very short life-cycles and very short development and production lead times are required … Customers want to be treated as individuals. This leads to a people intensive, relationship driven operation…" - An Introduction To Agile Manufacturing by Brian H. Maskell 

Agile methodology is more than just accepting the new process model and practices. It is more about mindset, the way we think, develop and deliver our product/services to the customers. You need to understand the basic concept and adopt it into your organization. At Iran Agile 2013, we focused on how to make this happen. 

Why attend?
For those who want to learn how to improve their work and their organizations. Being a community driven event, you are encouraged to share. To learn from others. To ask. Technical talks will give you new insights for your software craftsmanship development. 

Call for papers
If you think you have something valuable to share, we welcome you to come and share it with other experts. There are a lot of different topics including Agile adoption and transformation, successful case studies, managing customer requirement, engaging customer for collaboration, knowledge management, self organization as well as technical excellences.

The aim is to stay focus on the value creation as our motto in Agile 2013 is "Practical Agile and Nothing Else".

If you are interested, stay tuned with us for the upcoming news. 
Official Page - Iran Agile 2013 
#IranAgile2013

Agile Management: The Challenges, Dilemmas and The Ways Ahead

Recently, I gave a presentation on agile management in a webinar. In the first part of presentation the issue and invalid assumption behind Traditional process model in software industry introduced and in the next step the Empirical and iterative software process model described. in addition, other interesting topics in the area of agile management including constraint management, lean management, motivating people and self organization is also considered and finally some recommendation and useful resources suggested.

You can grab the slides here "Agile Management: The Challenges, Dilemmas and The Ways Ahead".

Finally I would like to send a special thanks to my dear friends in IrScrum specially Asad and Arash for all the help and kindness.

The challenges for Informatics in developing software for modern multikernel computers

Abstract

The purpose of this post is to examine the introduction of parallel computing and the challenges of software development for Parallel execution environment. First I will introduce the idea of parallel computing and up next I will present and evaluate the challenges of parallel computing along with their solutions and finally some conclusion will be drawn.

Vertical & Horizontal Development in Computing

The question arise when we are thinking about how the complex scientific problems of the twenty-first century including climate modeling, genomic research and artificial intelligence are testing the limits of the Von Neumann model of sequential processing.
 
In the past, computer scientists worked on the new approach to extend the power of computers in vertical manner, this means that they were working on producing huge super computers but with recent advances in technology and reducing cost of resources and arrival of multi kernel processing has helped us to think about new ways to solve huge and complex problem in parallel manners.

Introduction to parallel computing
 
For the most part, along with a host of new research questions that have arisen in the last decade, there remains a significant challenge today. Parallel processing offers the promise of providing the computational speed required to solve important large-scale problems. In fact, parallel processing requires a big shift in how we think to solve the problem.
 
Regardless of new hardware technologies, we should think about the new approach of developing software systems and also the way we think about our problem and presenting our solution. (Design and Analysis of Computer Algorithms).
 
Challenges of parallel computing

For the sake of applying the power and flexibility of multi-core processors, we should think about a new approach to breakdown huge problems into smaller elements. A better illustration of parallel processing occurs when a divide and conquer model is used to solve a task.
 
In this approach the problem is successively partitioned into smaller and smaller parts and sent off to other processors, until each one has only a trivial job to perform. Each processor then completes that trivial operation and returns its result to the processor that sent the task. These processors in turn do a little work and give the results  back to the processors that gave them the tasks, and so on, all the way back to the originating processor. In this model there is far more communications between processors.
 
n the next step, we should think about how to express our program which can be executable in a parallel computing environment. Functional Programming plays a vital role in this area, since it provide programmer to solve their issue in functional manner rather than sequential processing. There are simple principles in functional programming such as avoiding Mutable states, Lambdas, Closures and more importantly declarative paradigm which help programmers to free their mind about concurrency, synchronization, Race Condition and other multi core computation issues.
 
Although parallel functional programming helps us to represent our program in declarative manner in order to be applicable for parallel execution, but the problem is remain unsolved without thinking about how we can manage data in parallel computing environment.

 Industrial Revolution of Data – Age of Big Data

 We’re now entering into new age of computing named as “Industrial Revolution of Data”. In fact, the majority of data will be produced automatically by different kinds of machine such as software logs, video cameras, RFID, wireless sensors and so on.
 
Due to the considerable decrease in cost of computer resources, storing those data is so cheap, so companies tend to collect and store them in huge data warehouse for future when it can be mined for valuable information. The Big Data now comes to play, working with such distributed, huge and complex data would be impossible or better to say inefficient with existing software and databases system. 
 
We should think about other approaches for storing large set of data which is stored in different computers and in the next step effectively mining and executing queries from those sources. Perhaps the biggest game-changer to come along is MapReduce, the parallel programming framework that has gained prominence thanks to its use at web search companies.
 
The research in parallel computing has had the most success and influence in parallel databases. In fact, instead of breaking out a large problem into smaller element execute by different threads simultaneously, parallel database help us to store, querying and retrieve data from distributed resources over network effectively.

 MapReduce as Parallel Programming Framework

MapReduce algorithm is invented by Google to cope with Big Data in their search engine system. In fact, MapReduce is containing two simple primitives function which are available in Lisp and also in other functional languages. The computation include two basic operation, a map operation which execute on input records containing key/value pairs, and then invoking a reduce operation which collect and aggregate all responses from different nodes.
 
There are many different Implementations in different programming languages which are exist and used in industry for processing large set of data. In fact, most of NoSQL databases use this algorithm for collecting data from different sources in distributed heterogeneous environment. 

The biggest advantage of MapReduce is that it allows for distributed processing of map and reduction function. In fact, it allows us, to collect and process distributed data stored in different machine simultaneously.

 Conclusion

Parallel computing can help us to solve hug complex problem in more efficient way. In order to parallelize our task we should think about different challenges which we cope in developing software for parallel execution environment.
 
However, we should bear in mind that parallel computing is useful when we are facing with a big problem which can distributed among different computing agents. In addition, we should deeply think about the nature of problem, time as well as limits and costs of Parallel Programming.

The Management 3.0 - Leading Agile Developers, Developing Agile Leaders

For people who get the message, this book may prove to be as valuable as Darwin’s book On the Origin of Species.

This book introduce and evaluate different management theories and it will teach you how to use them in your agile environment. In fact, it is really a scientific book and recommended for everyone who wants to know more about agile philosophy from the scientific perspective.

 This book will discuss about General System theory, Game and Chaos theory as well as Evolution theory. In this book, Jurgen does a great job of explaining the science behind complexity and how Agile management methods have arisen from the need to manage in complex, dynamic, and unpredictable circumstances

Six Thinking Hats

While facing with a number of choices, we may find it hard to make a decision, or may always approach problems in the same way. Emotional people, for example, may not consider decisions calmly and rationally. Many successful people think from a very rational, positive viewpoint, and this is one reason for their success. Often, though, they fail to look at a problem from an emotional, intuitive, creative or negative viewpoint. By always using a positive approach, they may underestimate possible difficulties - such as resistance to their plans and be under prepared for dealing with further problems.

'Six Thinking Hats' is a valuable technique for increasing the effectiveness of decision making. Created by Edward de Bono, it makes you consider the decision from a number of perspectives, forcing you to add different ways of thinking to your usual approach.

This gives you a fuller view of a situation. As a result, your decision and plan will be ambitious, creative and sensitive to the needs of others. They will be carried out effectively, and you will be prepared for the unexpected. You can use Six Thinking Hats with other people or on your own. With others, it has benefit of blocking the confrontations that happen when people with different thinking styles discuss the same problem.

Each 'Thinking Hat' is a different style of thinking. With the White Thinking Hat you focus on the data available. Look at the information you have, and see what you can learn from it. Look for gaps in your knowledge, and either try to fill them or take account of them. This is where you analyze past trends, and try to work out from historical data what might happen in the future.

Wearing the Red Hat, you look at problems using intuition, instantaneous reactions, and emotion. Also try to think how other people will react emotionally. Try to understand the responses of people who do not fully know your reasoning.


Using Black Hat thinking, look at all the bad points of the decision. Look at it cautiously and defensively. Try to see why it might not work. This is important because it highlights the weak points in a plan. it allows you to eliminate them, alter them, or prepare contingency plans to deal with problems that might arise. Black Hat thinking helps to make your plans tougher and better able to survive difficulties. It can also help you to spot fatal flaws and risks before you start on a course of action.

The Yellow Hat encourages you to think positively. It is the optimistic viewpoint that helps you to see all the benefits of the decision and the value in it. Yellow Hat thinking helps you to keep going when everything looks gloomy and difficult.

The Green Hat stands for creativity. This is where you can develop creative solutions to a problem. It is an unstructured way of thinking, in which there is little criticism of ideas. A whole range of creativity tools can help you here.

The Blue Hat stands for process control. This is the hat worm by people chairing meetings. When running into difficulties because ideas are running dry, they may direct activity into Green Hat thinking. When contingency plans are needed, they will ask for Black Hat thinking, and so on.

Packt GlassFish Security book contest

About two weeks ago, I just participate in contest for winning Packt GlassFish Security book by Masoud Kalali, and the great news is that today I received an email which is mentioned that I'm one of the lucky winner of this contest.

I'm really happy and lucky for winning this prize and I would like to thank Masoud and Packt for this reward.

If you are interested about this book you can download free chapter "Designing and Developing Secure Java EE Applications" from Packt website. This book cover most important aspect of security in J2EE Application with GlassFish and additionally it will teach you how you can secure your Enterprise Application by using OpenDS and OpenSSO.

If you want to develop secure enterprise application with GlassFish, I would recommend to take look at this book. I'm sure that it will be helpful.

Best Practices for building Large Scale Agile teams

Building large teams, either traditional or agile, requires careful design. Bigger teams involve more communication, more decisions, more meetings, more documents, and, of course, more politics.

Agile principles of collaboration, simplicity, responsiveness, and minimally sufficient documentation can be applied to building larger teams but the application is far from simple. Building effective, efficient large agile teams takes a concerted effort in the four areas mentioned earlier: organizational design, decision-making design, collaboration/coordination design, and applying agile principles. 

Although collaboration, coordination, and knowledge sharing are critical to large projects, the downside of too much communications can be endless meetings and wading through tons of documentation and emails. Too little or poor communications, however, means no one understands the project nor their part in it. So whether we are doing organizational design or collaboration/coordination design, agile teams always balance on the side of "just a little bit less than just enough."

Organizational Design
In general, as projects escalate from large to very large the number of specialty teams will increase and a few members of the specialty teams will be full-time members.

In addition, as size increases, feature teams will be organized into a structure, product area team, capability team, feature team based on the product's component architecture.

Whereas in a traditional large-team hierarchy the teams might be functional (requirements analysts, developers, testers), large agile teams maintain their cross-functional and product orientation. Where very close coordination is required between feature teams (or between certain feature and specialty teams) cross-linking should be used in which one person from each team sits in on key meetings of the other.

Another design criteria is that a large agile team maintain the flattest structure possible (more nodes, less hierarchy).

This organizational structure focuses on the collaboration and coordination between autonomous but linked groups. As the team size increases to encompass several feature and specialty teams, there is a critical combination of team self-organization and self-discipline required. Individuals have responsibilities within a team structure, and teams have responsibilities within an overall project structure.

This network structure isn't a hierarchically controlled one, but neither is it a pure network structure in which all control is delegated to the nodes. The structure might be labeled a "modified network" structure in which a significant amount (but not all) of the power and decision making are distributed to the feature teams.


Collaboration/Coordination Design
In Agile Software Development, Alistair Cockburn discusses various communications modalities and indicates each one's relative effectiveness.

For example, a two-way face-to-face discussion at a white board is more effective than sending a document to someone. However, for large and especially distributed teams, the enemy of effectiveness remains cost. Although it may be very effective to have a face-to-face design discussion at a white board, the cost of doing it when the individuals are located 5,000 miles apart may be prohibitive, although new Web 2.0 collaboration and project management tools are continually reducing these costs.

So the real question is not just effectiveness, but cost-effectiveness for the particular job at hand. What makes communications design so difficult is that it rests on a foundation of relationships of trust, respect, appreciated cultural differences, and shared context. 

A team that is "in sync" can get away with lower effectiveness communications modalities because they have a good relationship context for sharing information. Another team distributed across countries in which there is little respect, trust, or multi-cultural understanding will need higher effectiveness communications modalities.

This also means that the collaboration/coordination design will probably change as projects progress and relationships become better established. Communication modalities that might not work early in a project may work fine toward the middle and end.

Two factors in designing collaboration/communications practices and tools for a team are relative effectiveness of the modalities and the foundation of trust, respect, and understanding of different cultural norms.


Decision-Making Design 
How is a large agile team different from a large traditional team from an organizational perspective?

• First, a large agile team has a flatter, less hierarchical structure fewer layers of managers.
• Second, to the greatest extent possible, decisions are pushed out to the feature teams or specialty teams.
• Third, feature team members participate in specialty teams to ensure their input is heard and to take part in the decisions.
• Fourth, team decision making, whether project or technical decisions are being made, are accomplished in a participatory fashion.
• Fifth, specialty teams are encouraged to issue guidelines, not fiats, and furthermore (like managers) they are encouraged to make as few decisions as possible.
• Sixth, peer-to-peer (feature team to feature team) interactions and dependency management are encouraged.
• Seventh, the team embraces agile principles.


Knowledge Sharing and Documentation
Documentation and knowledge sharing are two important issues in scaling agile projects. A 6-person team needs different documentation practices than a 100-person team does.

The Agile Manifesto states, "Working products over comprehensive documentation." Large, front-loaded projects that spend months, and even years, gathering requirements, proposing architectures, and designing products are prone to massive failures.

Documents support communication and collaboration, enhance knowledge transfer, preserve historical information, assist ongoing product enhancement, and fulfill regulatory and legal requirements. They are not unimportant, just less important than working versions of the product.

Understanding comes from a combination of documentation and interaction, or conversation, the conversations among people who have domain knowledge. Furthermore, as the complexity of the knowledge to be transferred increases, the ability of documentation alone to convey that knowledge decreases and the need for interaction with knowledgeable people increases dramatically. Documentation provides content (partially), but conversations are necessary to provide the necessary context.

A little research into the field of knowledge management shows that early efforts to document best practices (software development, engineering, or otherwise)—filling reams and reams of web pages with details—has generated marginal benefit. In thinking about knowledge transfer, one must distinguish between transferring explicit knowledge (written down) and tacit knowledge (in someone's head). "

Tacit knowledge cannot be transferred by getting it out of people's heads and onto paper," writes Nancy Dixon in her book Common Knowledge "Tacit knowledge can be transferred by moving the people who have the knowledge around. The reason is that tacit knowledge is not only the facts but the relationships among the facts—that is, how people might combine certain facts to deal with a specific situation."

In summary, documentation should be designed in a lean, barely sufficient manner, both formal (for retention) and informal (temporary), highly visual and visible, viewed as support for collaboration and coordination, and vary considerably by project size and type (regulated environments for example). Ultimately the issue isn't documentation but understanding.


Self-Organizing Teams of Teams
An agile team consists of individuals who interact within a structure of self-organized and self-disciplined rules of engagement. Individuals have a degree of autonomy within this loose structure, and they, in turn, exercise self-discipline to be accountable for results and behave as responsible and thoughtful members of the team.

Larger agile teams, those consisting of multiple feature and specialty teams, operate the same way, individuals are the agents in teams, whereas feature teams are the agents in a larger project.

In building large agile teams, a network replaces the common hierarchical structure, decision making and collaboration must be carefully designed, and discipline reflects rules of engagement across teams.

Team Self-Discipline
Just as individuals have responsibilities to their teams, teams themselves have to be self-disciplined to work within a larger self-organizing framework.

The behaviors required of teams closely parallel those for individuals:
• Accept accountability for project results.
• Engage collaboratively with other feature teams.
• Work within the project self-organizing framework.
• Balance project goals and team goals.

A team that is unwilling to work within the established framework disrupts the work of the larger project in the same way that individuals who are unwilling to work within the team framework do.

Teams have to align their own goals with those of the project. There will always be too much to do and too little time, and the tendency will be to work on one's team goals rather than on project goals.


Process Discipline
We all know the saying "Don't fix things that aren't broken." For larger teams, we need to give this advice a twist: "Don't always fix things that are broken." Although smaller teams are not immune to excessive process, this tendency becomes more prevalent as teams get bigger.

Just as agile teams don't try to anticipate future requirements or design, choosing instead to let them emerge over time, they shouldn't attempt to anticipate every problem and put processes or practices in place to prevent them. It's often cheaper and faster to fix an actual failure than to spend excessive time anticipating failures that may never occur.



references:
- Cockburn, Alistair. Agile Software Development. Boston: Addison-Wesley, 2006.
- Highsmith, Jim. Agile Software Development Ecosystems. Boston: Addison-Wesley, 2002.
 - Rueping, Andreas. Agile Documentation: A Pattern Guide to Producing Lightweight Documents for Software Projects. New York: John Wiley & Sons, 2003.
-Dixon, Nancy. Common Knowledge. Boston: Harvard Business School Press, 2000.