Presentations of the 2 Surveys about the ISO/IEC 29110 series
A first
survey was conducted in 2006 and a second survey was conducted in 2018
by members of the ISO Working Group 24. The main objective of the 2006
survey wasto question VSEs about their
utilization of ISO/SC7 standards and to collect data to identify problems and
potential solutions to help them apply standards and become more competitive.
The main objective of the 2018 survey was to obtain feedback from VSEs,
their customers, auditors and professors about their utilization of the
ISO/IEC 291100 series.
Survey conducted in 2018
In 2018, a survey was conducted to obtain feedback from VSEs, their customers, auditors and assessors and, from educators.
Objectives of the Survey
In addition to gather general information from respondents (e.g.,
demographic data, type of software/system developed, standard used),
the other objectives of the survey conducted in 2018 were:
o From Very Small Entities (VSEs)
- Resources needed and problems encountered to implement
- Raisons for using or not using the ISO 29110
- ISO 29110 standard and/or guide implemented
- Benefits obtained from implementation and certification
- Difficulties encountered during implementation or certification
- Satisfaction of the implementation
o From customers of VSEs
- Satisfaction of their suppliers using ISO 29110
o From auditors/assessors of VSEs
- ISO 29110 standard or guide used for certification/assessment
- Difficulties encountered
- Weaknesses observed in VSEs
o From academia
- Using ISO 29110 in courses
- Using ISO 29110 in students’ projects
- Reasons for teaching or not teaching ISO 29110
- ISO 29110 standard and/or guide used
- Research project about ISO 29110
- Material needed to teach ISO 29110
- Difficulties encountered when teaching ISO 29110
- ISO 29110 Certification obtained
- To obtain suggestions to improve the ISO 29110 series:
o To improve the actual portfolio of ISO 29110 standards and guides
- e.g. number of processes, activities, tasks and work products
o To suggest new standards or guides
- e.g. about security, measurement, DevOps
Results of the 2018 Survey
Satisfaction of VSEs
VSEs
were asked if they were satisfied with overall ISO/IEC 29110
implementation. As illustrated in the figure below, over 88% of the
respondents indicated that they were completely or largely satisfied.
Only 11% indicated that they were partially satisfied. VSEs were also
asked if they noticed improvements to quality and productivity:
improvements in productivity (64%), in quality (58%) and process (73%)
were observed in the first six months after the implementation of
ISO/IEC 29110. Only 4 respondents (7%) did not observe an improvement
to productivity or quality.
Satisfaction of VSEs about their ISO/IEC 29110 implementation
The figure below illustrates how long it took to notice improvement to their productivity.
Time to notice improvement to productivity with ISO/IEC 29110
The figure below illustrates how long it took to notice improvement to their quality.
Time to notice improvement to quality with ISO/IEC 29110
Satisfaction of Customers of VSEs
The
figure below illustrates the satisfaction of the customers of VSEs as
suppliers. Over 79% were completely and largely satisfied with their
suppliers and 16% were partially satisfied.
Satisfaction of customers of their suppliers that have implement ISO/IEC 29110
Satisfaction of Academia
The figure below illustrates the motivation of Professors to teach or use ISO/IEC 29110 in students' projects.
Motivations to teach or use ISO/IEC 29110 in students’projects
The figure below presents the management and engineering guides that are taught or used for projects.
Management and engineering guides taught or used in students’projects
The
figure below illustrates that the ISO/IEC 29110 guides have been used
by academia for a few years. A few professors have been teaching the
ISO/IEC 29110 for up to 9 years.
Number of years that ISO/IEC 29110 has been taught or used in students’projects
The figure below illustrates that a few teaching organisations obtained a formal ISO/IEC 29110 certification.
Number of teaching organisations that obtained an ISO/IEC 29110 certification
The figure below illustrates that a few universities develop software
for internal and external customers with ISO/IEC 29110. One university,
that has obtained a formal ISO/IEC 29110 certification, has started to
sell the software developed in its certified environment.
Software developed by academia for internal and external customers
Survey conducted in 2006
In 1997, the Technical Council on Software Engineering
responsible for IEEE Software Engineering Standards (SES) initiated a survey to
capture information from software engineering standards users in order to
improve those standards. They gathered 148 answers, mainly from the USA (79%)
and large companies (87% of them having more than 100 employees). The purpose
of this section is not to systematically compare the two sets of survey results.
However, even though IEEE survey objectives differ from those of the ISO/IEC
survey, there are some interesting common findings. The IEEE survey underscores
the fact that ISO/IEC standards are often used in organizations rather than IEEE
standards. IEEE survey respondents also indicated that IEEE standards need to
be improved, mainly by adding examples and templates and a life cycle process
definition, and by providing support for metrics and measurement.
The WG24 survey was developed to question VSEs about their
utilization of ISO/SC7 standards and to collect data to identify problems and
potential solutions to help them apply standards and become more competitive.
From the very beginning, the working group drew up several working hypotheses
regarding VSEs. The survey was intended to validate some of these hypotheses,
such as the following:
-
The VSE context requires light and well focused life cycle
profiles
-
Particular business contexts require particular profiles.
-
There are significant differences, in terms of available
resources and infrastructure, between a VSE employing 1 to 10 people and an
Information Technology (IT) department of the same size in a large company.
-
VSEs are limited in both time and resources, which leads to
a lack of understanding of how to use the standards for their benefit.
-
Benefits for VSEs may include recognition through
assessment or audit by an accredited body.
The survey questionnaire and an
introductory text were developed by the WG24 and translated into 9 languages:
English, French, German, Korean, Portuguese, Thai, Turkish, Russian and Spanish.
The survey is made up of 20 questions structured in 5 parts: General
information, Information about standards utilization in VSEs, Information about
implementation and assessment problems in VSEs, Information about VSE needs and
Information about justification for compliance to standard(s).
A Web site, hosted by the École
de technologie supérieure (ÉTS), was
developed to maximize the number of responses and facilitate data collection and
analysis. A mailing list was created using WG24 members' contact networks. We
also contacted centers and software engineering professors dedicated to small
software enterprises.
Respondents were informed that
it would take a maximum of 15 minutes to complete the survey. They were informed
that all data would be kept confidential, and that only summary results and
project data that could not be matched to a specific VSE would be included in
the published results.
In
order to increase participation in the survey, WG24 promised to send all
respondents a report presenting, anonymously, the survey results. The survey was
launched in February 2006, and, as of October 2006, over 432 responses had been
collected from 31 countries.
|
Country |
Number of Responses |
Country |
Number of Responses |
|
Argentina |
2 |
Italy |
2 |
|
Australia |
10 |
Japan |
3 |
|
Belgium |
10 |
South Korea |
4 |
|
Brazil |
70 |
Luxembourg |
3 |
|
Bulgaria |
3 |
Mexico |
20 |
|
Canada |
10 |
New Zealand |
1 |
|
Chile |
1 |
Peru |
4 |
|
Colombia |
109 |
Russia |
4 |
|
Czech Republic |
3 |
South Africa |
10 |
|
Dominican Republic |
1 |
Spain |
3 |
|
Ecuador |
9 |
Taiwan |
1 |
|
Finland |
13 |
Thailand |
58 |
|
France |
4 |
Turkey |
1 |
|
Germany |
1 |
United Kingdom |
2 |
|
India |
57 |
United States |
3 |
|
Ireland |
10 |
|
432 responses collected from 31 countries
Results of the 2006 Survey
VSEs expressed the
need for assistance to adopt and implement standards. More than 62
percent indicated that they would like more guidance with examples, and
55 percent asked for lightweight and easy-to-understand standards.
Concerning the weak use of standards by VSEs, the following reasons
predominated: a lack of resources second, that standards are not
required and the nature of standards themselves.
More than 15 percent of the respondents considered that the current set
of software engineering standards (e.g. ISO/IEC/IEEE 12207) are
difficult and bureaucratic and do not provide adequate guidance for use
in a small business environment.
Finally, a three-quarters majority of VSEs felt it was important to be
evaluated or certified against a standard to increase competitiveness,
customer confidence and satisfaction, software product quality,
facilitation of marketing and higher potential to export and to
decrease development risk.
ISO working group 24 used the data of the survey to develop a set of 58
requirements, as illusstrated below, that have been used to develop a
series of software and systems engineering standards and guides.
