About Deploy Project
Workpackage 11 — Measurements
The measurement workpackage aims at measuring as objectively as possible the impact of the deployment of advanced formal
engineering methods on dependability and productivity. In the high level terms, productivity and dependability may be defined as follows:
- Dependability is an integrated concept defining system ability to deliver a service that can be trusted and encompassing reliability,
availability, safety, and security where the weight of these four attributes varies across domains and projects.
- Productivity is the ability to provide competitive products in terms of functionality, time to market, effort and quality.
Measurement in DEPLOY is split in five tasks. The first two tasks T11.1-T11.2 determine the measurement methodology, while tasks T11.3-T11.5
define a measurement coordination strategy so that measurement effort from each industrial deployment WP (within task TX.6 of each of them)
learn and benefit from each other
- T11.1-T11.2 - Development of a methodology for measuring dependability and productivity.
The methodology will allow customisation to the specific domains. It will be based on the state of research in software process
measurement adapted to the use of formal methods and on the state of measurement practice at the industrial partners. Specific metrics
for measuring formal specifications will also be identified.
- T11.3-T11.5 – Measurement coordination. The first task will set up the measurement coordination. It will defining the means
to collect the data, starting from simple spreadsheets that might later evolve in a more elaborated data collection system (simple to
use, with less overhead, more consistency check, reminders,…) It will also enable to collect the data in a coordinated way across all
projects and to keep a DEPLOY-wide consistency in the measurement process for example for impacting changes in the measurements (such
as new measurements or enhancements). Such adaptations are expected between the first and the second deployment. The actual measurements
will take place within TX.6, both for the pilot and enhanced deployments. DEPLOY-wide trends will be analysed within WP12 after the
completion of project specific analysis. Efforts will also be devoted to make the methodology reusable beyond the end of the project
as a practical tool for managers leading similar projects.
Organisation and scheduling
WP11 will follow the individual roadmap of each industrial deployment.
- During the global set-up phase, the main task will be devoted to the methodology definition based on the state of research
and practices, including the identification of specific metrics for formal specifications. This part will require the collaboration
of all industrial partners to ensure that the measurement methodology fits the industrial constraints. It will also be validated by
the research partners during a project meeting to make sure it follows the overall DEPLOY strategy and that it correctly captures
dependability and productivity issues.
- Starting from the knowledge transfer period, the methodology will be instantiated into a specific measurement process for each
industrial WP (within task TX.6). The instantiation will require tailoring measurements taking care to address the peculiarities
of each industrial partner because of differences in the development processes, the nature of the organizations and sector specific issues.
- Measurement data specific to each project will be gathered during the life-cycle of the pilot deployment (estimated to take place in year 2).
The same process will be repeated as much as possible on the reuse deployment (estimated to take place across years 3&4).
The coordinated measurement framework will ease the collection and mitigate the risk of overlooking data.
- Draft reports will be updated continuously as the measurements proceeds. Sharing of anonymous measurements will take place after
the end of the pilot study, and used to produce a first consolidated document. A similar process will occur for the enhanced study.
Conclusions will then be drawn from those two iterations and across all the industrial domains. Given the sample size of a few pilot
projects, conclusions cannot be overgeneralised. However, it is expected that they will identify global trends, highlighting the areas
where formal method are most successful, giving the rationale behind it, identifying problems and suggesting possible adjustment for