2018-data for the Student Survey is published in February

About the Student Survey

Studiebarometeret is a national student survey that are sent to more than 60 000 students each autumn. The survey asks for the students perceptions of educational quality in their study programs. The purpose of Studiebarometeret is to strenghten the quality work in higher education and give useful information about educational quality. On Studiebarometeret.no you can easily:

  • find the students' evaluations of their study program
  • compare results for different study programs
  • see the development over time for each study program

The portal is useful for applicants, students, institutions and members of staff and others with an interest in  higher education. New results are published in February each year. Studiebarometeret is initiated by the Ministry of Education and Research and conducted by the Norwegian Agency for Quality Assurance in Education (NOKUT).

 

The web portal

How to use the portal?

You can compare students’ opinion of the quality of different study programmes. From the front page, you can find different study programmes by:

  • Searching for specific study programmes, university colleges/universities, or campuses
  • Choosing one of the eight academic disciplines (represented by graphical symbols), or one of the underlying subfields

The search results will give you an overview of all study programmes matching your search criteria. Select individual study programmes to see their results. Index values are shown at the top of the page, while results for individual questions can be found by scrolling down. Responses range from 1 to 5, with 5 being the highest score.

What is new?

We have implemented several changes in the new version of the portal in 2017. Previously, you could only use the search engine on the front page to look up individual study programmes by name. It is now possible to search for specific institutions and campuses. Campus is now also included as a filter in the Student Survey to indicate the location of the programme, and is shown to the right of the institution name.  In order to filter results by campus it is necessary to first select an institution.

We have also made it possible to download data from the portal to Excel. You can download data by adding the study programmes you are interested in to the “Your chosen study programmes” tab, and then clicking “export all to Excel”. On the article page Download data sets, http://studiebarometeret.no/no/artikkel/5 , you can also download data for all programmes available in the portal.

We modify the survey every year. In 2016 we introduced three new question batteries: Internationalisation, Use of digital tools/media, and Student expectations. Some of the batteries from 2015 were discontinued, while some individual questions were rephrased. A consequence of this is that responses from a portion of the questions cannot be compared across surveys. For more information about changes to the survey, please see the “Student survey” header below.

What kind of information can you find at Studiebarometeret.no

The Student Survey (Studiebarometeret) contains students’ assessment of 1800 study programmes. Students answer each question using a scale from 1-5, where 5 is the highest/most favourable score. For each study programme, the Student survey contains the following information:

  • The average score for each question, calculated from the answers from students attending the specific study programme. Results are shown both as indexes (groups of related questions) and for individual questions. You can read more about indexes below.
  • Detailed results for each question for each study programme, including distribution, number of respondents and standard deviation.
  • Historical data for up to the four most recent surveys. We do not show time series for programmes where we have aggregated response data.
  • Average results within the corresponding subject group the study programme belongs to.
  • Facts about the study programmes. These include number of credits, total number of students, whether the study programme offers work placement, teaching language and organisation (campus-based, decentralised, web-based).
  • Link to the web page of the HEI offering the specific study programme.

Threshold values and merging results

Threshold values for publication in the web portal

NOKUT’s threshold value for publishing data in the web portal is over 9 respondents and a response rate of more than 19.5% for each study programme. Alternatively, the programme can have between 6 and 9 respondents if the response rate is over 49.5%. Threshold values have been put in place to ensure that published data are robust, and that students who have answered the survey remain anonymous.

Merging results

Many study programmes are very small, and consequently seldom reach the threshold values. In order to increase the number of study programmes shown on studiebarometeret.no, we merge results from the last two years if the programme had at least one respondent in the most recent year. If the combined results cause the programme to reach the threshold value, we publish results from the programme in the portal.

On studiebarometeret.no we indicate that data have been merged with the text “the numbers for 201# and 201# have been merged due to too few respondents in 201#”. When numbers have been merged, the reported number of respondents and response rates are based on both years. However, we do not show time series for merged data, as the programmes do not reach the threshold value for a given year.

 

Merging results from 2015 and 2016

In 2016, 48% of the study programmes had 20 or fewer recipients, making it hard for many of the programmes to reach the threshold values. Based on 2016 data alone, 54% of the programmes had enough respondents for publication of results. After merging with the 2015 data, we could publish results for 73% of the programmes.

Changes in the questionnaire

Due to changes in the questionnaire (from year to year), we are not able to show results for all questions when we merge data. Omitted questions are marked with a ‘-‘ in place of response values. For more information about changes to the questionnaire, see the text below “The student survey”.

About indexes

Studiebarometeret contains seven indices showing aggregated results for questions on related topics. These are: learning environment, student participation, inspiration, vocational relevance, teaching, student evaluation and learning outcomes.  You can see which questions are included in each index by clicking the index name on the results-page of studiebarometeret.no.

The results score for each index is the average of the score for the individual questions included in the index. Questions are weighted equally. Questions are included in an index based on thematic relatedness and on results from statistical analyses.

If the questionnaire goes through considerable changes from one year to another, it is not possible to compare the indices over time. In such cases, we provide a short explanatory text instead of numerical scores in the viewer for “historical data”, with a dash (“-“) for each individual question.

Subfields and subject groups, and calculating averages

The web portal groups study programmes according to disciplines, subfields, and subject groups.

These categorisations correspond to Statistics Norway’s (SSB) Norwegian Standard Classification of Education (NUS). Every study programme is categorised according to the NUS coding scheme, with disciplines as the topmost category. The eight main discipline categories are displayed on the front page of studiebarometeret.no. Each discipline contains several subfields, Narrow fields (shown beneath the discipline on the front page), and subfields are in turn divided into subject groups. The coding scheme also places study programmes according to degree level.

For instance, 3-year nursing programmes belong to the discipline Health, welfare and sport, the Narrow field Nursing and caring, and the Detailed field nursing studies. Master’s degrees in nursing studies are placed in corresponding categories, but at the Master’s degree level. Corresponding Bachelor’s and Master’s degrees are consequently placed in the same discipline, Narrow field and Detailed field, but at different levels. A Bachelor’s degree in nursing is coded 661120, while a Master’s degree in nursing science has the code 761104. The first digit indicates the degree level, the second indicates discipline, the third indicates Detailed field and the fourth digit indicates the Detailed fields. The two last digits, in combination with the four preceding ones, indicate specific study programmes. For example, all the 3-year nursing programmes in Norway have the same code.

NOKUT receives information about the study programmes’ NUS-code from the institutions, and only changes these when there are obvious errors. NUS-codes are sent to NOKUT ahead of each survey. 

We present averages for the type of education the programme belongs to, in the following way:

  • If the programme belongs to a subject group with a sufficiently high number of programmes, and with enough respondents, we show the averages for the programme’s Detailed field.
  • If there are too few programmes with enough respondents in the Detailed field, we show the averages for the relevant Narrow field.
  • If there are too few programmes with enough respondents in the Narrow field as well, we show no score averages.
  • In order for us to display score averages (for fields), at least four study programmes need to reach the threshold value for publication in the web portal.

Only Bachelor’s programmes are used to calculate averages for a field at the Bachelor’s degree level. This applies equally to Master’s programmes. Consequently, national averages will differ across degree levels within a field.

Student's workload

Students estimate how they allocate their time by writing numerical values in text fields in the Student Survey. As a result, these questions differ from those in the rest of the survey, where students can only answer along a more limited scale (1-5, or “do not know”). The text fields did/do  not allow the input of values above 80, and NOKUT does not remove outliers (extreme values) or manipulate the data in any way.

NOKUT’s data on study programmes includes information about whether programmes are full- or part-time.  Part-time students will usually spend less time on studies compared to students enrolled in full-time programmes. The web portal indicates each programme’s “percentage of full-time” as part of the information about workload.

In 2013 we phrased these questions differently. Consequently, it is not possible to compare numbers for 2013 with numbers from subsequent years, and we do not display 2013-numbers for time series.

 

Questions on vocational training

In 2014, NOKUT included a set of questions concerning students’ experience with vocational training in the survey.

Who have had vocational training?

From 2016 and onwards, we ask all students in the survey whether they have had vocational training (yes/no). Only respondents who select “yes” receive the question on their experience with vocational training.

«Vocational training» refers to study programmes with vocational training periods determined by the National Curriculum and/or supervised vocational training. The vocational training has to be a part of the study programme, and the institution must be responsible for quality and supervision. We explain this to respondents by way of an “information button”.

Criteria for displaying vocational training responses in the web portal

The same group of students may have differing opinions on what constitutes vocational training, and consequently on whether or not they have had vocational training.

To ensure that only responses from study programmes where the students have actually had vocational training are included for these questions, NOKUT uses the following selection criteria: a minimum of six individuals in the study programme must have answered at last three out of seven vocational training questions, and these individuals must comprise a minimum of 49.5% of those who have answered the Student survey. That is, at least half of the respondents in a programme must have selected “yes” for the opening question “Have you received vocational training organized by your study programme?” Additionally, at least six of these respondents must have answered a minimum of three out of seven question on vocational training. The criteria is crosschecked with other available information, and is robust.

Programmes without data on vocational training in the portal

For study programmes that do not have scores for vocational practice, we display the text «Not enough data/no vocational practice training” in the portal.

Solutions prior to 2016

Previously, we asked institutions directly which of their study programmes included vocational training. Based on the institutions’ responses, questions on vocational training were sent to students in the relevant study programmes. This method had a number of weaknesses, and was subsequently replaced by the selection criteria described above, which has been carried over to the 2016 survey and subsequent surveys,

Why did NOKUT create the Studiebarometeret?

NOKUT administers and maintains the national Student Survey and its associated web portal, studiebarometeret.no, as mandated by the Norwegian Ministry of Education and Research. The purpose of the portal is to provide concise and user-friendly information about students’ opinion of the quality of education offered at Norwegian higher education institutions. This information is meant to help strengthen the further development of quality in higher education, and to provide applicants with useful information that can help them to choose between different study programs and campuses.

The target audience for the Studiebarometeret includes:

  • Institutions offering higher education
  • Higher education applicants
  • Students
  • The Government
  • Other parties interested in higher education

One of NOKUT’s core mandates is to promote quality in higher education. NOKUT’s work is also meant to help ensure that society as a whole can be confident in the quality of Norway’s higher education. The Studiebarometeret is an important tool for promoting and spreading knowledge of the quality of education.

Contacts

If you have any questions or comments about the Studiebarometeret, feel free to contact us at Studiebarometeret@nokut.no. Alternatively, you can contact the following persons directly:


 

Ole-Jacob Skodvin

Director of Analysis and Development

Phone: +47 21 02 18 29

            

Pål Bakken

Senior Advisor, Department of Analysis and Development

Phone: +47 92 08 46 71

             

Magnus Strand Hauge

Advisor, Department of Analysis and Development

Phone: +47 90 10 34 28

 

Lars Fredrik Pedersen

Advisor, Department of Analysis and Development

Phone: +47 41 28 12 04

 

The Survey

The questionnaire

The survey contains questions about students’ opinion on the quality of different aspects of their study programme. Questions concern the study programme the student is enrolled in, not the institution or specific courses. All students receive the same questionnaire, independent of their study programme. The survey takes approximately 10-12 minutes to complete, provided the student does not provide extensive comments when answering. Students can complete the survey in Norwegian, English, or Sami (from 2016).

Topics

  • Education and academic supervision
  • Study- and learning environments
  • Student participation/influence (?)
  • Inspiration
  • Vocational training
  • Work relevance
  • Student assessment
  • Learning outcomes
  • Expectations
  • Student engagement, workload, and motivation
  • Teaching and study methods

Studiebarometeret - all batteries 2017 - ENGLISH

Studiebarometeret - without voluntary batteries 2017 - ENGLISH

Changes in the questionnaire

We continuously seek to improve the questions in the survey, in order to make them as precise and succinct as possible. At the same time, in order to preserve the basis for comparison and analysis over time, we wish to keep the questions as similar as possible from year to year. You can see an overview of the survey’s core questions displayed in bar charts on studiebarometeret.no/en/.

We keep 3/4 of the questionnaire the same each year, while 1/4 can be changed. This means that we can take whole question batteries (sets of related questions) in or out of the questionnaire, depending on what we wish to focus on in a given year.  This system, using alternating batteries, also means that we can ask the same questions at different intervals (for example on a biennial basis). Changes are made in consultation with the education sector, for example through our consultation group. In 2015 a working group, consisting of three participants from the Norwegian Association of Higher Education Institutions (UHR), contributed to changes in the questionnaire with the aim of improving the survey’s usefulness to quality assurance work at the institutions. We also evaluate questions in interviews with students.

Changes in 2017

Most of the questionnaire was left unchanged. However, NOKUT did make a few alterations, based on feedback from institutions and the consultation group as well as our own analyses. Based on the feedback, NOKUT created new questions and conducted a validation of the new question. This was done by an electronic survey among students (670 respondents) and two focus group interviews with students. The validation was valuable and led to some adjustments. The survey for 2017 is about the same length as the survey for 2016, and shorter than the survey for 2015.

The following changes were made:

  • Norwegian Agency for Digital Learning in Higher Education (NUV) and NOKUT have developed a new inventory of questions about the use of digital tools in higher education. The questions are different from last year’s survey. Last year’s survey had an additional survey about the use of digital tools and media in education. This addition will not be included this year.
  • Proba research is conducting a survey on barriers in higher education for students with disabilities. The Student Survey for 2017 will contain a question asking the students if they are willing to participate in the survey from Proba research.
  • The Norwegian Centre for International Cooperation in education (SIU) and NOKUT have developed several new questions about internationalisation in higher education. These questions replace last year’s questions on the same topic. The questions about internationalisation are filtered based on the student’s responses. For example, Norwegian students who have studied abroad will get different questions from international students studying in Norway.
  • For 2017, the institutions can select up to two inventories of questions from a list of four additional/selectable inventories:
    • A) Extent of feedback and academic supervision
    • B) Satisfaction with feedback and academic supervision
    • C) Teaching and learning methods – usage
    • D) Teaching and learning methods – contribution

    The first two inventories were also part of the Student Survey in 2015. The last two inventories have been a standard part of the Student Survey for the last three years. If the institution chooses not to include any of the four selectable inventories, the survey will be shorter than last year’s survey.

  • The inventory “study environment” from last year’s survey has been split in two. The questions remain unchanged, except for one question, which has been moved to a different inventory (“organisation of the study programme”).
  • There is a new inventory called “organisation of the study programme”.
  • The inventory “student assessment” has had a major change. Only two of the five previous questions has been kept, and we have introduced four new questions.
  • Time spent on academic activities: The explanatory text has been changed.

Changes in 2016

Most of the questionnaire was left unchanged in 2016. However, NOKUT did make a few alterations, based on feedback from institutions and the consultation group as well as our own analyses. We added some questions and removed other. In sum, however, the questionnaire was somewhat shorter than in 2015.

The major changes for 2016 were as follows:

  • In relation to a pilot project, in cooperation with Norwegian Agency for Digital Learning in Higher Education (Norgesuniversitetet), we added a battery of questions on the use of technology in education. Additionally, we offered respondents the option of answering an extra set of questions from Norgesuniversitetet’s survey «Digital tilstand», accessed through a separate link (see under question battery 16 in the questionnaire).
  • We added a battery of questions on internationalisation. The questions were developed in cooperation with the Norwegian Centre for International Cooperation in Education (SIU). These questions mostly measure the extent of internationalisation rather than the quality, as opposed to most of the other questions in the survey.  The questions are therefore not directly comparable to results from other batteries.
  • The two batteries of questions on «scope of feedback and advising» and «satisfaction with feedback and advertising» were taken out. The battery on “importance for overall assessment of the study programme” was also removed.
  • Questions on "students’ expectations regarding their studies" from 2015 were changed to "(student’s impression of) expectations from academic staff".
  • We made some changes in the battery on working life relevance. Consequently, the score for the index variable was lower compared to previous years: we included a question on cooperation with workplaces in the labour market, and combined two questions on knowledge and skills into one question on workplace relevant competence. Our assessment is that the changes have improved the index, and has ensured that it covers the topic of working life relevance better than previous versions.

Changes in 2015

The major changes to the questionnaire in 2015 were as follows:

  • We removed three question batteries: Reasons behind choice of study programme, Student exchange abroad, and Work relevance (whether your job is relevant to your studies).
  • We included four new question batteries: Scope of feedback and academic supervision, Satisfaction with feedback and academic supervision, Expectations, and Importance for overall assessment of the study programme.
  • We removed three questions on Time allocation: (time spent on) Studies in total, Unpaid work related to studies and Other unpaid work.
  • We added two questions on Study- and learning environments: The relationship between students and academic staff and Administration and information.
  • NOKUT exchanged the term «teachers» with «academic staff», and provided clarification of the terminology: “By academic staff, we are referring to teachers, teaching assistents, lab assistents and other academic staff contributing to teaching.”

Population

Participation in the national Student Survey is voluntary for both institutions and students. Our goal is to include all study programmes at the bachelor’s and master’s degree level with a duration of at least 3 years. This includes 4-, 5- and 6-year vocational education such as teaching programmes, 5-year lecturer programmes, civil engineering programmes, medicine, veterinary medicine, etc.

The survey also encompasses more or less all Norwegian institutions offering bachelor’s and/or master’s degree programmes. A small number of private institutions and military academies do not participate.

The survey goes out on an annual basis to 2nd-year bachelor’s students, 2nd-year master’s students, and to 2nd and 5th-year students enrolled in master’s programmes with a duration of five years or more. The survey also goes out to students in 4-year teaching programmes. Students on leave of absence are not included in the population.

Every year we send the survey to students in their 2nd (5th) year of study on their programme. Consequently, we do not follow the same individuals over time.

In autumn 2017 the survey encompassed approximately 64 000 students across around 1800 study programmes.

Execution

The survey is conducted annually during October-November. We publish the results on studiebarometeret.no in the beginning of February the following year.

NOKUT collects students’ contact information and data on which study programme they belong to from the institutions. Each student receives a link to their individual questionnaire via email and SMS. Additionally, students may log in to their questionnaire either by password, via links provided on learning platforms such as Fronter and It’s learning, and via social media such as Facebook. The survey can be answered on PCs, tablets and smartphones.

After collection, the response data is mined, quality tested, and adapted for publication.

Response rate

In 2016, approximately 29 000 students answered the questionnaire, comprising 45% of the population. The response rate was 47% in 2015, 42% in 2014, and 32% in 2013.

The response rate ensures that we can show results for 73% of the study programmes participating in 2016. Displayed results include study programmes with combined data from the last two years.

NOKUT’s threshold values for publication of data is set at over 9 respondents and a response rate above 19.5%. Alternatively, a study programme can have between 6 and 9 respondents, provided the response rate is above 49.5%.  It is the larger study programmes that manage that manage to satisfy the publication criteria to the greatest extent. Consequently, over 90% of all students are enrolled in the study programmes for which we display results in the web portal (73% of all participating programmes in 2016). With few exceptions, excluded results are mostly from smaller study programmes. The population contains a significant number of study programmes with 10 or fewer recipients (of the survey), making it hard for these to reach the threshold values.

For study programmes with few respondents in the most recent year, we combine the results with those from the preceding year. A number of the programmes will then have enough respondents for them to be displayed in the portal.

Response rates at the institutions

At the institutional level, the response rates vary greatly – ranging from 23 % to 83 % in 2016. Most of the largest institutions have a response rate of around 40%.

On Studiebarometeret.no we display the number and and response rate of respondents for each study programme. The higher the response rate for a study programme, the more representative the results are for students in that programme.

Combining data

In order to display results for a higher number of study programmes, NOKUT combines data from the current year with that of the previous year in cases where a programme does not reach the threshold values for publication in the portal. We display results for these programmes, provided that the combined numbers satisfy the publication criteria outlined above. Study programmes with lower response rates can still be looked up and viewed on studiebarometeret.no, but will not be displayed with results from the survey. With few exceptions, this mostly occurs with small study programmes with 10 or fewer recipients (of the survey).

With combined data from 2015 and 2016, we can display results for 73% of the approximately 1800 study programmes on studiebarometeret.no. Among these programmes, an average of 23 students completed the survey. Average response rates for these programmes were 53%.

 

Further development

NOKUT works continuously in order to improve and further develop the implementation of the national student survey and studiebarometeret.no.

We appreciate all feedback on potential improvements! For questions or comments about the Student Survey, please contact us at: Studiebarometeret@nokut.no