Quora answer: What is software architecture?
http://www.quora.com/What-is-software-architecture
There are quite a few good books about Software Architecture. So I think the literature defines Software Architecture well. But what you are really asking about is where the Architect fits into the Software Development Team. With the change over from Agile and Scrum approaches to Software Development the architecture and requirements seems to be left out of account in most of the agile process models and so it seems that the architect role and requirements role have been eclipsed and we seem to think that architecture and requirements are no longer needed and systems can accrete, but of course this leads to Technological Debt. So what I would like to address is the agile at scale where the project is big enough to need the differentiation of requirements and architecture to produce a coherent product.
Now the problem seen with Architecture and Requirements is that they produce documents that are not executable, and thus get out of date, and are not kept up throughout development, and thus are seen as waste from a lean perspective. And as you say all developers have a view of the whole system they are working on in their heads, so why do we need architecture? I am going to try to address that question.
We should recognize that Design itself is applicable at all levels of System and Software development. Thus developers who do not consider themselves or are not considered “architects” are doing design when ever they create something new or different in software. We have software patterns, but the combination and adoption of the patterns is still a design task, even if we do not have to make up new patterns. Design is ubiquitous in development and no one has exclusive access to design within software development, and especially if we adopt agile methods, we should realize that everyone owns the design, and autocratic approaches that give design rights to one developer over others may be expedient but in the long run is harmful, because everyone on the team needs to share the architectural design of the product they are developing together. We know how to share implementation, but we are not as good at group self-organization of the architecture. But the self-organization of the team should reflect the self-organization of the architecture.
I would like to mention that I have just done my dissertation on Emergent Design and what I am going to say is based on that research See http://about.me/emergentdesign
The problem is with agile (scrum) type methods is that although they espouse self-organization of the team there is no real theory of self-organization upon which they are based. Therefore I would like to offer a model of self-organization upon which I will base my remarks.
See http://www.mediafire.com/?jfkjoe2bkddky6a
See also http://www.mediafire.com/?y122wwv7d89qp5u
I have constructed a model of software process based on this model in my book Wild Software Meta-systems. http://works.bepress.com/kent_palmer/
So given that context let me begin by saying that self-organization can be seen in the nature of knots that are organized against themselves by their own self-interference. So we have a very precise model of self-organization in knot theory. Knot theory is a new discipline, and probably one if the youngest of the mathematical disciplines. The best work on this is that of Louis Kauffman http://www.math.uic.edu/~kauffman/ whose work I follow. Particularly because he is interested and takes seriously the work of G. Spencer Brown in Laws of Form. http://www.lawsofform.org/
Self-organization is related to self-production which is Autopoiesis, and we are talking about teams that produce these complex systems in most cases so that means that we have reflexive autopoietic systems exhibiting this self-organization. Self-organization of the team, and the organization of the product developed by the team are two different but deeply related matters. What we need to understand better is how the alleopoietic organization of the product of the team reflects the self-organization of the team.
In other words we are arguing that requirements and design are intrinsic to the self-organization of the team, and that this is reflected in the requirements and architectural design of the product produced by the team. So let us consider what requirements and how they relate to functions. Requirements have to be developed in relation to a functional model of the system. This is because without the functional model you would not know whether the requirements set is complete, consistent, or clear or balanced, or fully describes the intended system. Requirements are ideally Godellian statements of the hypothetical emergent properties of the system that is desired. I like the statement that I read somewhere that the requirements carry the customer value of the system being developed, and allow us to keep our focus on that during the development process. But Requirements are un-ordered, they are axiomatic statements.
The physical and functional architecture are related to the Agent and Functional viewpoints on the realtime system which are partially ordered. But those partially ordered viewpoints ultimately have to yield embodiment in spacetime as eventdata computation that is fully ordered. Thus the relation of requirements, design, and implementation are determined by what Klir in Architecture of Systems Problem Solving, the methodological distinctions with respect to the ordering of variables in the system.
Now what is fascinating is that once we realize that the methodological distinctions orders determine the level of organization which is present in the system being developed and that there is a duality in this lattice between linear order without distance or partial order with distance. These two duals have the kind of duality that exists in the minimal methods. The minimal methods are the information order that is necessary to capture the relations between viewpoints. Since there are four viewpoints on a realtime system there are six different transformations between these viewpoints. Interestingly these more or less correspond to what exists as methods in UML and SysML. Functions were taboo in the original UML specification but this minimal method has been added back in with SysML. The only difference between object oriented and functional dataflow minimal methods is whether you are looking at data from the point of view of function or function from the point of view of data.
Now once we understand the minimal methods as bridges between the viewpoints on a real time design we can start thinking about this in both allopoietic and autopoietic ways in order to relate the self-organization of the team and the alleopoietic organization of the architecture of the system.
My own research led me to create the Integral Software Engineering Methodology (ISEM) which was a domain specific language to give a model based description of software architectural designs. Recently I gave a paper on how this might be updated to work at the Systems Design level based on the research in my dissertation. This paper can be seen at http://kentpalmer.name which is my resume page. But at the bottom you can see my CSER paper on the changes that I have made to my design language to make it easier to do Systems Design.
First I advocate using Domain Specific Languages to express design. You can see why in my critique of SysML: http://holonomic.net/sml01a03.pdf.
The key idea that instead of diagrammatic visualization that we get in UML and SysML we should express designs in text like we express code. This way as text we can keep it like the code and even put it within the code as comments. It could act as constraints on the code structure and thus be part of the execution strategy when we build the code or even be the basis of code generation. The problem with UML and SysML is that they are semantically weak being composed of only relations.
So let us go back to the question at hand, which is how the self-organization of the team is related to the organization of the structure of the software. Architecture can be seen as a protocol for organizing team coordination over the structure of the product being built. In other words the team provides a reflexive social meta-system within which individual team members function together and coordinate their development work. But part of the information they have to convey to each other and preserve is about the internal structure of the application that they are building. Requirements and Architectural Design is part of the protocol for communicating about the structure of the program that is based on the self-organization of the team. In other words the team is producing an intersubjective synthesis. The production process is structured and that allows the product to be structured at the macro level. But this means there must be a protocol which will allow the team to develop different parts of the system without working against each other or stepping on each others work, such that the whole system works together when integrated, verified and validated.
If we see requirements and architectural design as this communication protocol that holds the macro synthesis together then it is possible to see how architecture and requirements are nothing different from the value stream of the team by which the emergent properties sought are projected into Being. If one is doing software within some predetermined framework then the framework is giving the order to the whole application that the various team members are contributing to. But if the application has a unique structure that needs more than the framework then we will need an architecture that is designed to give a macro structure to coordinate work.