(CONTINUED FROM 16/10/10 )
2.8. Initiatives’ impact: measurements and observations
The study originally aimed at identifying innovative and impactful e‐Inclusion case studies from the private and no‐profit sector in order to capture the current situation in term of initiatives coming from these two sectors and to understand how the Commission could channel efforts, support practitioners and export experiences. However, with approval from the Commission, we were forced to modify the purpose of the research insofar as evidence of initiatives’ impact could not be observed a priori and case studies could not be identified and short‐listed on the basis of their degree of impact. This is mostly because e‐Inclusion initiatives tend to be promoted locally by small‐ or medium‐sized organisations that often do not recognise the need to measure quantitative outcomes of their projects. It has also been observed that even when impacts are measured, seldom are these used to communicate the success of the initiative externally, including potential users. However, it is common for organisations to use data to track their own progress and/or to report success to financing entities.
Therefore, despite the progresses achieved by the study and despite the high level of analysis carried out on the 13 case studies chosen, it remains difficult to make generalisations and to provide quantitative evidence on impacts. Nevertheless, we attempt here to provide examples of good outcomes linked to the characteristics we have used throughout the study to analyse initiatives. More specifically, as shown in Table 3 below, the vertical axis indicates the aforementioned characteristics, whilst the horizontal axis provides an assessment of the degree of impact. Hereafter, a discussion for each topic is provided with examples from the case studies and their impact measurement exercises.
Table 3: Summary of impacts
Source: authors’ elaboration 23
Although the choice of initiatives for the purpose of this study was carried out so to keep a balance in terms of size of organisations involved and level of intervention, between national and local initiatives (see Figure 3), in‐depth analysis presented in Annex highlights that small local initiatives tend to be more highly recognised by the community of reference, which is turn more likely to recognise the benefits of the initiative itself. Users feel more enthusiastic about technology novelties when they can pull down the barriers between them and ICT, when they can refer to an intermediary, when they can combine technology with a personal interest; in other words, when they can meet other people and feel part of a community. This is why local initiatives seem to have generated more impact on users: they are more manageable and they have a much stronger hold on their community, which makes it easier to reach potential targets.
A perfect example is Computer Clubhouse in Ireland, through which Intel’s research found out that members who visit more frequently and stay longer:
• Scored higher in breadth and depth of technology use;
• Expressed more engagement in school; and
• Scored higher on measures of problem‐solving competences.
Another example is IBM’s Liberated Learning: through each university the Consortium has managed an optimal implementation of the speech‐recognition technology and has reached a good number of students in need of support. Unfortunately, also due to privacy restrictions, the Consortium practitioners have not been collecting information on the ultimate impact of their initiative, such as time saving or improved students’ performance.
However, it has been further observed that what makes local initiatives more effective is the replication of similar experiences elsewhere on the regional or national territory. This is the case of Wifi Village, Log On Learn, MoLeNet. Molenet, for instance found that a comparison of the (mostly predicted) retention data for nearly 5,000 FE college learners (approximately half the total 2007/08 MoLeNET learners) with LSC national in‐year retention rates for 2006/07 suggests an improvement in retention of 8. Furthermore, a comparison of the (mostly predicted) achievement data for nearly 5,000 FE college learners (approximately half the total 2007/08 MoLeNET learners) with LSC national in‐year achievement rates for 2006/07 suggests an improvement in achievement of 9.7%.
Moreover, practitioners from Wifi Village found that out of all the people involved in the project 22,5% of the participants had already looked for a job, compared to 7% of people interviewed for the control group, thus showing a much higher inclination to joining the labour force within people who acquire IT skills.
Imagination, enthusiasm, and relentless pursuit of a vision are therefore invaluable in the small‐scale grassroots types of initiatives that commonly serve the constituencies not easily reached by government or bigger organisations.
The case studies analysed for the purpose of this study also provide numerous examples of how broad‐based partnerships with involvement of private and no‐profit stakeholders are critical to success of e‐Inclusion initiatives. The cases studied seem to suggest that initiatives that have a multi‐stakeholder approach are more likely to generate successful impacts. This could be the case of the Virtual Bus. Moreover it is generally observed that the formula for success involves NGOs as leaders in the implementation process, whilst private‐sector partners make the partnership strong thanks to their financial support that guarantees sustainability and to their advisory ability that ensures appropriate resource allocation as well as provide expertise on different matters.
This is the case of BT’s and Scope’s Wheeltop Project, where the staff working with the students at Beaumont College feel they see the students become more actively engaged with the curriculum as a result of taking part in the project. Examples include being able to type their own work independently rather than dictating, being able to operate a video camera more independently rather than directing staff to do so, being able to access and contribute to the college blog on their own devices rather than relying on support and being able to perform choreography using specially designed, accessible grids rather than expressing ideas for transcription by staff. Unfortunately the students do not get given definitive grades for their work so it is difficult to keep track of their performance at school in a more quantitative way.
The broad involvement of both an NGO and a large corporation has ensured that the initiative comprised people with insights into the needs of the target group as well as people with insights into the financial and managerial aspects of the initiative. Therefore, it is crucial that e‐Inclusion initiatives have the ability to raise support and funding independent of government, both in kind such as specific knowledge regarding location‐ or user‐specific circumstances, as well as in the form of capital, equipment, and infrastructure.
In terms of field of intervention, if we cross the case studies with the information we have gathered on results, which indicate small/medium and local initiatives to be the most impactful, we would be led to conclude that the highest results have been achieved by projects that focus on Learning through ICT and eAccessibility. MoLeNet, LogOn Learn and Computer Clubhouse are examples of the first case, whilst Reasonable Adjustment Scheme and Wheeltop are illustrations of the second case.
With particular regard to Learning through ICT, several projects focus on creative applications of ICT such as audiovisual and web design as a way to achieve a double dividend: to transfer high‐level skills while keeping retention and participation, which is always a challenge in job insertion schemes. It appears that creative applications are both in demand from the labour market and attractive and stimulating for the participants to the training. Projects such as Mode83 is a main proof of the benefit of this approach: absence rates are almost zero, while they are normally quite high in other work integration projects, and around half of the trained people find a job after the training. The Clubhouse has a similar approach for school drop‐outs.
While the impacts of some of these cases have been aforementioned, the Reasonable Adjustment Scheme launched by Barclay’s found that in 2005 80% of employees agreed that ʹmanagement supports equality and diversity in the workplaceʹ (75% in 2004). Furthermore, the number of people identifying themselves as disabled in the Employee Opinion Survey has gone up by 22% since 2004. This has allowed Barclay’s, not only to deal with disability in a more appropriate manner, but also to create a more welcoming and comfortable working environment for its employees. Barclays was voted 27th out of 80 companies in the first ever Employers Forum on Disability benchmarking exercise; Barclays Capital US was named Employer of the Year by the National Business and Disability Council.
However, analysis of our case studies has led to conclude that the most impactful cases are those focusing on Networking and Advocacy activities. Despite not measuring its ultimate impact, UK Online Centres for instance can currently count about 6,000 affiliates – of which 3,000 are libraries, about 1,000 are represented by educational institutions and 2,000 by NGOs. Furthermore a staggering 1.3 million courses had been carried out and completed by a total of 250,000 users, across an overall supply of 30 modules. Because organisations like UK Online Centres and the newly born Telecentre Europe bring together a high number of existing local initiatives while at the same time creating new ones, they become channels of a multiplier effect that enhances and disseminates their knowledge of user needs, their expertise and resources.
Finally, in terms of tools of implementation, another central element is to ensure that measures are tailored to the circumstances of an individual. Some initiatives have specifically aimed at the development of advanced digital literacy skills and there are some examples of initiatives where participants have learned to use ICT in more advanced and interactive manner (see for instance the Virtual Bus). This emerging advanced deployment of ICT is enabled by increasingly affordable technologies such as mobile phones, iPods, and PDAs (this is the case of MoLeNet), through collaborative functionalities called WEB 2.0 and through a open source software platforms, for example Facebook (see for instance Wheeltop Project). These technologies can be used for a range of purposes: from looking for support in an online community, to a group of elderly who share a common interest about historical events in their community.
These initiatives stress the importance of understanding user needs as a prerequisite to tailoring initiatives to individuals’ circumstances. The promotional methods used to motivate and attract new participants are also varied and range from “word of mouth” ‐ often through the many volunteers ‐ to newspaper articles, advertisements, webpage promotion and even radio and TV campaigns (see for instance UK Online Centres). These emerging changes in the use of ICT must certainly be acknowledged in the ways e‐Inclusion is understood and addressed. 26
(TO BE CONTINUED)
Annalisa De Luca