Second, ― _Who will be evaluated”?_ (A fresh class of graduates or those who have already a few years‘ experiences). Given the fact that many fresh engineers change jobs within a year or so, external effects such as in-house training should be removed as much as possible in order to assess learning outcomes at institutions. It was therefore decided that the target population would be the fresh graduates from technical and engineering institutions for whom this was their first job. Hence, employers were asked not to consider engineers for whom this was not their first job after graduation.

Third, ― _At what level should the employers evaluate?_ (At individual level, at institutional level such as Indian Institutes of Technology, National Institutes of Technology, institutions that participated in the Technical Education Quality Improvement Program, etc, or by overall average of all fresh hires)‖. It was decided to send one single questionnaire to each company due to the administrative burden to deal with multiple questionnaires per different graduate group. Employers then evaluated all first-time-employed engineers hired over the previous four years together as one group.

Fourth, ― _How will the questionnaires be distributed to companies?_ (Online survey, email invitation, or paper-based questionnaire via physical mail)‖. The on-line survey was finally selected as the survey method because it is easy to manage and organize the data collected from employers. Further, on-line survey can be easily used again in the next round of the employer satisfaction survey in two years. FICCI randomly selected employers who were provided with a username and password. After logging into the survey, employers were asked to complete the survey and provide company characteristics.

# **3.3 Survey Instrument**

The questionnaire design builds upon three sources: the expected learning outcomes used for accreditation by the National Board of Accreditation (NBA), previous employer surveys, and consultations with employers. The questionnaire is divided into three sections; (i) Overall satisfaction level, (ii) Importance and Satisfaction of 26 different skills, and (iii) Employer‘s characteristics.<sup>4</sup> Employers were asked to evaluate both the importance and satisfaction levels of each of the 26 skills on a five point scale.

The NBA, India‘s only official accreditation body for engineering education, has established 11 Program Outcomes. NBA is a provisional member of the Washington Accord—an international agreement between accreditation agencies for engineering education for 18 countries. Therefore, NBA‘s program outcomes (expected learning outcomes for graduates) are based upon the internationally agreed set of the skills and knowledge that graduates are expected to possess at the time of graduation.<sup>5</sup> The NBA criteria are:

> 4 The survey questionnaire is attached in Annex 3. The questionnaire asks about importance and satisfaction for 26 skills divided into two overall groups: General and Specific skills. The General skill referred mainly to personal characteristics while Specific skills mainly referred to those skills directly related to technical and engineering professions as well as communication and computer skills. The categorization of skills into General and Specific Skills was conducted in an a-priori manner following consultations with government officials, employers, and academia. Section 5 will go in detail more systematic and empirically-based categorization.  The survey questionnaire is attached in Annex 3.

> 5 The NBA learning outcomes and the ABET learning outcomes are very similar, but do have a few important differences. For example, the NBA criterion (e) asks for the graduates to ―demonstrate an ability to

8