[A previous post about Productivity Intelligence and 3D analysis is here. You can also see a presentation and read the use case. This article has been posted in Eclipse Newsletter – Project Quality – September 2014.]
What Productivity Intelligence is.
Engineering Group has adopted an innovative approach within its Software Labs to measure quality in development processes. This approach is named Productivity Intelligence for two reasons.
The former is related to the meaning of “productivity”. It does not simply refer to lines of code written per day or man-hours worked per day! It’s not just the economic efficiency of company employees. It’s about integrating all the dimensions involved in code production. I mean adding to the two traditional dimensions Economic – development costs and efficiency – and Technical – software measures, bug fixes, test coverage – the Social one: customers’ and developers’ satisfaction. This allows to give stage to the voice of the users as well as of developers.
The latter refers to a technological reason: the adoption of business intelligence techniques to monitor and improve productivity, through integration of data coming from different sources. This helps to efficiently integrate various development tools in a common infrastructure collecting different production information. It’s about building bridges between different technologies and development practices.
I’m not a developer
My current focus is not on building solutions, but on building integrations. Probably it’s because I’m quite aged – working in the IT field from a very long time – or probably it comes from my attitude in community building in open source. I’m currently working to help create an environment letting developers proudly do their best and meet the corporate requirements at the same time. This helps to bridge the gap between developers and enterprise standards and offers a more comprehensive vision of how software development performed under “quality effectiveness” practices can significantly improve the results.
This article focuses on my current experience in a large industrial software factory where I’m trying to combine developers’ expectations with project managers’ needs and corporate procedures through an integrated approach in order to achieve the right level of quality performance. At the same time, I’m trying to combine agility with effectiveness.
The enterprise point of view
A software company must provide its customers with software at the right level of quality, producing it at the right level of economic and organizational efficiency, in compliance with consolidated quality standards, most of time using state-of-the-art technologies and, finally, bringing innovation into the Information Technology market.
The pressure for standardization is very high, in particular if your development process must comply with quality certifications such as ISO, ITIL and CMMi standards and you must provide both your certification authorities with the evidence of the continuous improvement of quality practices through an effective measurement process, and your top managers with the continuous enhancement of productivity.
The developer’s point of view
This opens a wide debate: what is a developer and, mainly, a good developer? If I knew the answer, I could give him the right support and tools. Once again, am I facing this question from the developer’s point of view or from the enterprise point of view?
In any organization you can find many types of developers: workhorses, good programmers, rockstars and great programmers. We must balance all talents and attitudes, but let’s stay in the middle now.
A good programmer wants to release good code. It is measured by evaluating what he puts out, instead of what he knows. He is brilliant and reliable, he stands for simplicity and excellence. He can inspire his team to do the same. A good programmers is competent and writes “solid” code. This is not “clever code”: it’s well-organized, adopt standards and use comments, it is reusable and well-tested. A good programmer does things in a straightforward way.
But a really good programmer carries an abiding passion for programming. It is competent, wants to achieve the expected outcomes and strives to give its contribution to the team. It’s open to new approaches and refines its skills by learning and testing new technologies, trying out new languages, exploring new tools and reading about programming. He likes to solve challenging problems and finds his satisfaction in finding the best solution for every requirement.
At the end, the real question is not “what a good programmer is” but “what a good programmer needs”. It’s about giving stage to developers’ voice.
The need of a different point of view
According to my open approach, which I have adopted long time ago, I know that you cannot impose a development model, but that you can influence its affirmation. You must be able to integrate different points of view, which follow.
The economic one – it’s obvious; moreover only commoditizing repeatable tasks, can an enterprise focus on funding new and brilliant developments.
The technical one, because programmers want to release good code and the enterprise must adopt good development processes.
The social one, meaning listening to end users, to integrators that are enhancing our products and giving stage to the voice of project managers as well as developers.
It’s about collecting both quantitative and qualitative data that can be used to measure performances in order to actually measure my quality and productivity.
Consequently, at Engineering Group I’m currently promoting a new approach to measure productivity. Engineering’s Software Labs (ESL) include hundreds of software engineers and developers committed to providing enterprises and public administrations with software applications based on state-of-the-art technologies and consolidated quality standards, bringing innovation into the Information Technology market. Nowadays this market demands agility, i.e.: reduction of delivery times, iterative cycles, effectiveness and quality.
ESL has developed a specific platform which includes processes and tools supporting the effectiveness of project management and Application Lifecycle Management (ALM) activities, also integrating the infrastructure of the Quality Assurance (QA) department. The general infrastructure supports the continuous improvement of quality practices, the measurement and enhancement of productivity, as well as the assessment of development processes in compliance with quality standards, such as ISO and CMMi.
The technological solution
The technical infrastructure is based on a portal integrating a set of applications allowing to manage project documents, risks, the traceability and development of requirements, testcases, through reports, graphs, statistics and dashboards, supporting an effective activity monitoring.
This allows the immediate use of dedicated tools and to retrieve updated information supporting the historical data analysis. At the same time, this information becomes part of the corporate repository of lessons learned in the project management domain.
The requirements traceability and development are managed through a project management tool, which is free for open source projects (Atlassian Jira). Its flexibility and the availability of a powerful workflow engine facilitate the implementation of the processes defined by the QA department for Requirements Management and ALM, through general processes that are applicable to any new project, without necessarily reinventing the wheel every time. Thanks to this tool, the communication among the project management environments and the development ones has been enhanced. This improves the task notification and effort estimation processes and the recording of the time spent on each activity, which the developers can perform through the user-friendly interface of their development tool.
In fact, the Integrated Development Environment (IDE) is Eclipse. In particular, adoption of eclipse Mylyn solve the context switching issue, reducing information overload and making multitasking activities easy. This allows the programmer to focus every time only on the current requirement, connecting all the related source code, to make impact analysis and facilitate handover of development tasks.
Fig 1 – MyLyn screen
The integration with a versioning system (Subversion) allowed an effective information traceability, from the requirement definition to the realisation of the final work product.
Thanks to the adoption of a flexible model, the QA department can use the same project management tool to track internal audits on projects. The information on each performed audit is linked to possible issues raised during the process, as well as the instructions to solve them. This way, the QA department has full traceability of its activities, shared with the addressees of the audits, ensuring project and process improvement.
Testing activities are managed through a specific open source tool (TestLink) that is integrated with the project management tool, allowing you to link requirements to test cases and to record bugs, test cases and test results .
The system is monitored by Spago4Q, which is an analytical application of SpagoBI -the open source business intelligence suite. Spago4Q is a vertical solution supporting the measurement, analysis and monitoring of products, processes and services. It allows to build reports, graphs, statistics and dashboards on a single or a set of projects, enriched with KPIs to monitor project performances, retrieve information from different sources and integrate them in coherent meta-models.
Fig 2 – Spago4Q architecture
All analytic views are built with SpagoBI Studio – a report and dashboard development environment based on Eclipse.
Fig 3 – SpagoBI Studio
Moreover, a specific Jira connector, leveraging Jira REST API, was realized to allow the easy realization of eclipse BIRT reports, that are integrated in SpagoBI, eliminating any need of interaction with databases and the correlated security concerns.
Spago4Q collect measures from the previously described tools and provides performance values on three dimensions of analysis: Economic, Social and Technical. It allows the software factory to measure and increase the productivity level, combining quantitative data (economic and technical measures and metrics) and qualitative data (customers’, integrators’ and users’ level of satisfaction), according to the QEST 3D model.
Qualitative data are collected by means of an open source survey software tool (LimeSurvey) where respondents’ answers are evaluated adopting the Net Promoter Score (NPS) approach, using Six-Sigma Transfer Functions.
Fig 4 – QEST dashboard
The iterative definition, collection and analysis of multidimensional measures at each life cycle phase offers the feedback required to make adjustments to the project processes in a timely fashion, both for the next phase and for designing future improvements to the process of the preceding phase. Drill-down capabilities provide both a unified view of global performance, detailed views on single processes, as well as performance comparisons.
Fig 5 – QEST drill-down
The current list of goals defined for each dimension of analysis follows.
- Economic dimension: improve productivity, reduce maintenance effort, reduce rework, improve development resource allocation, optimize usage of hardware resources
- Technical dimension: reduce resolution time for defects and technical issues, improve software delivery time, improve quality of testing processes, improve quality of source code
- Social dimension:, improve software compliance with corporate standards and procedures, improve function reuse, evaluate improvements on training skills for organizational resources, improve the level of satisfaction of customers, integrators and developers, foster knowledge sharing.
All the tools listed in this paragraph are just a selection of a wider list of tools currently used at the Engineering Software Labs. The technological infrastructure grants high level of flexibility and scalability to integrate different tools, also to process the same activity (e.g.: a different tool for project management or a new tool for code quality analysis).
The theoretical support
Any model should be reliable. Consequently, a theoretical support is requested. Very few open source solutions deal with the measurement, monitoring and control of projects, accomplishing with the requirements from well-known models such as CMMi or ISO 15504 (aka SPICE). Also, only limited experience has been done in using open source tools to achieve a real, comprehensive, measurable quality level in OSS development itself.
The technological infrastructure described above implements QEST nD model , a performance measurement model, according to its three-dimensional (3D) implementation (Quality factor + Economic, Social and Technical).
Fig 6 – QEST model
The general reference model is PMAI, which enables the constant monitoring and assessment of the level of quality and cost-effectiveness of software development processes and practices. Specifically, the process includes four steps:
- Plan: define dimensions and metrics of analysis
- Measure: collect data to measure global performance value
- Assess: analyze results through business intelligence tools
- Improve: identify bottlenecks, solve issue and define areas for improvement.
Achieved goals include the continuous improvement of quality practices, the measurement and enhancement of productivity, as well as the assessment of development processes in compliance with quality standards, such as ISO, ITIL and CMMi.
Building bridges
The use of an integrated, low-cost and mostly open source solution, which can be easily extended and integrated with other software development tools, has been a key success factor at Engineering Group, fostering the adoption of well-defined ALM processes and an effective software lifecycle management.
This solution can integrate other applications and tools, reducing the duplication of information and fostering the sharing of lessons learned. Moreover, it can be used in different contexts, such as supporting the trustworthiness of information provided by open source communities and projects.
The innovative approach of this multi-dimensional analysis of the productivity – in a wider sense – offers access to complete knowledge of software production, its quality and of the human relations involved in the development phase. Standard models like QEST that are not bound to a specific context are suitable to a large variety of heterogeneous software productions environments.
At the beginning of this article I outlined my role in building integrations. This can be implemented both in bridging existing gaps in project quality activities and in building bridges between communities and organizations. In this article I mentioned tools provided by different open source communities or vendors. Open source is evolving, IT technology is evolving as well. To manage this, you must adopt an open approach, continuously monitor what’s happening in the market, make the right choices and be ready to make a “turnaround” when necessary.
This has a lot to do with openness, collaboration and innovation.