A comparative analysis of the cognitive levels of Science and Mathematics secondary school board examination questions in India

A comparative analysis of the cognitive levels of Science

and Mathematics secondary school board examination

questions in India

Anirban Roy Chowdhury

IISER Pune

[email protected]pune.ac.in

Nandagopal K S

IISER Pune

[email protected]

Vijay Prakash

IIT Bombay

prakashvija[email protected]

Sean Fogarty Syaamantak Das

IIT Bombay

syaamantak.das@iitb.ac.in

ABSTRACT

This research study investigates the cognitive levels of the

questions used for assessment and evaluation purpose in

three national school board secondary school leaving exam-

inations in Mathematics and Science subjects, conducted

in Indian schools from 2011 to 2020. The research used

Bloom’s Taxonomy to identify the cognitive levels of 3071

board examination questions. The study addresses the gap

in the current literature on the non-availability of a compar-

ative study of national school board assessment practices in

India. The study provides a comparative analysis of three

English medium Indian school boards (ICSE, CBSE, and

NIOS) by analyzing what areas/topics the board exams test

and which ones they ignore. Based on this analysis, certain

trends were analysed which are present in boards/subjects

in the 10th standard national level exams in India. This

study will help school education stakeholders to get an in-

sight into the cognitive level trend/patterns of assessment

questions.

Keywords

Cognitive level, Assessment Questions, School Examination,

Indian School Boards

1. INTRODUCTION

A learning process can be segmented into three sections

[7]. First, the learning goals, which are also referred to as

instructional objectives / Learning Objectives (LO). They

specify the learning outcomes that mention the skills and

knowledge which is to be imparted to the learner. Second,

the selection of learning materials that meet the requirement

to achieve the learning goals. Third, assessment questions

that determine whether the learner is learning the necessary

skills and knowledge to solve the problem. In the educa-

tional domain, the Learning Objective refers to the state-

ment(s) of a course curriculum, speciﬁcally describing the

skills and knowledge that the student must gain after the

course is completed. If complemented by a similar cognitive

level of instructional techniques and tests, the learning out-

comes help the teachers determine whether the students are

achieving the expected skills and knowledge.

The Learning Objective also determines the performance

results by determining the conditions under which perfor-

mance will occur. It also deﬁnes the requirements (speciﬁc

skills, competencies, and attitudes) that the learners will

follow. “Learning outcomes are precise statements of what

faculty expects students to know and to be able to do in

some measurable way as a result of completing a program,

course, unit, or lesson” as stated in [3]. It was also observed

that “in addition to guiding, teaching, learning, and assess-

ment strategy, eﬀective learning outcomes facilitate student

orientation to the subject and communicate expectations”

as reﬂected in this research [6]. For active learning to take

place, ‘there must be a constructive alignment of the cur-

riculum, which should ensure that in an education program,

the learning objectives, teaching and learning methodolo-

gies, and assessment techniques should complement each

other’.

Student’s ability to think for activities during instruction

and examinations are essential for improving their intellec-

tual abilities, performance execution, and professional growth.

As a result, exams should be designed to encourage students

to express their thoughts on the exam questions, develop cre-

ative answers, and connect the exam answers to their own

experiences and real-life situations [5]. Also, good questions

not only promote eﬀective learning and assessment, but they

must also be consistent with curriculum and instruction, as

assessment has a signiﬁcant impact on both learning and

teaching. The phenomenon is known as the ‘washback ef-

fect’, and it refers to how testing aﬀects teaching and learn-

ing [17].

As a result, writing high-quality exams that include both

higher-level questions (HLQs) and lower-level questions (LLQs)

is critical in assisting students in achieving the desired learn-

ing outcomes and evaluating their level of proﬁciency in a

speciﬁc course. The HLQs help students dig deeper into the

learning materials while also encouraging critical thinking

A. R. Chowdhury, N. K. S, V. Prakash, and S. Das. A comparative

analysis of the cognitive levels of science and mathematics secondary

school board examination questions in india. In M. Feng, T. K

aser,

and P. Talukdar, editors, Proceedings of the 16th International Con-

ference on Educational Data Mining, pages 509–514, Bengaluru, In-

dia, July 2023. International Educational Data Mining Society.

under the Creative Commons Attribution NonCommercial NoDeriva-

tives 4.0 International (CC BY-NC-ND 4.0) license.

https://doi.org/10.5281/zenodo.8115778

and creativity. A work by [2] stated that, schools must em-

phasise higher-order skills in order to develop critical think-

ing.

Therefore, the present study seeks to examine and compare

to what extent the questions of the tenth grade school board

examinations in Mathematics and Science subjects prepared

by three national level Indian school boards include both

higher and lower-order thinking levels. In the ﬁrst place, it

is essential to assess how well students master the informa-

tion of the educational materials within the six levels of the

Bloom’s Taxonomy. Additionally, it is essential to analyze

whether the exam questions of the given examinations in the

Mathematics and Science subjects are based on both higher

and lower-order thinking levels.

2. LITERATURE REVIEW

2.1 Mapping Bloom’s Taxonomy Cognitive Lev-

els to Thinking orders

Bloom developed a taxonomy in this context that is used

to develop assessments that take into account each of the

six levels of hierarchy in the cognitive domain [14]. Knowl-

edge (recalling details), comprehension (description in some-

one else’s words), and application (using existing knowledge

to produce results) are examples of lower-order thinking

domains. Higher-order thinking domains include analysis

(discovering connections between facts and concepts), syn-

thesis (creating new original work), and evaluation (judg-

ing and demonstrating one’s position) [12]. Researchers

used Bloom’s Taxonomy’s two cognitive categories to anal-

yse and determine the levels of questions asked in exams, and

they established two types: lower-level questions (LLQs)

and higher-level questions (HLQs) [16]. The LLQs are de-

signed to test students’ recall of fundamental and universal

concepts and processes. The HLQs, on the other hand, are

more advanced and diﬃcult because they require students

to engage in deeper and analytical thinking processes.

2.2 School Board Examination System in In-

dia

The work by [8] gives a comprehensive overview of Indian

school education system. The research article extensively

covers the Indian school education system, which is one of

the largest education system in the world. With both pub-

lic and private schools, the Indian school system can be di-

vided into four main categories – pre primary (consisting of

pre-school, lower and higher kindergarten), primary school

(standard one to ﬁve), middle school (standard six to eight),

secondary school (standard nine and ten) and high school

(standard eleven and twelve / or pre university standard).

The public schools are majorly either central government

schools (such as Kendriya vidyalayas, navodaya vidyalayas,

Sainik schools etc.) or state government schools of respective

states. The private schools are usually run by individuals,

trusts or societies and may or may not receive fund from

the government. Apart from these two major categories,

some other semi government type schools run by local gov-

ernment bodies also exist (e.g. Municipality schools). The

central government schools are usually aﬃliated to the Cen-

tral Board for Secondary Education (CBSE) supervised by

the National Council of Educational Research and Training

(NCERT) under the Ministry of Education. The Council of

Indian School Certiﬁcate Examinations (CISCE) is a semi-

private, non governmental education board in India. It con-

ducts the Indian Certiﬁcate for Secondary Education (ICSE)

examination (for tenth standard) and Indian School Cer-

tiﬁcate (ISC) examination (for twelfth standard) in India.

These two are the major all India based school examination

boards. Apart from the all India based school examination

boards, the state government aﬃliated school examination

boards constitute a major part of Indian school examination

system. All Indian school boards for the sake of collaborat-

ing and exchange of information with each other forms an

umbrella body called Council of Boards of School Educa-

tion in India (COBSE), a voluntary association of all the

boards of school education in India. There are more than

50 members with associate members from Nepal, Mauritius,

Bhutan, Pakistan and United Kingdom. Other than these

Indian school boards, foreign school boards such as Inter-

national Baccalaureate Organization (IBO) and Cambridge

International Examinations (CIE) are emerging as newer

school boards in urban areas. These schools boards oﬀer

global school level examinations all over the world and fol-

lows universal curriculum. It must be noted that all school

boards in India are autonomous having their own syllabus,

curriculum, method of assessment and evaluation.

An elaborate research work on the quality of school educa-

tion in India has been done by Institute for Studies in In-

dustrial Development as mentioned in [9] for Quality Council

of India, New Delhi. Firstly, they have clearly deﬁned the

distinction between syllabus and curriculum. Curriculum is

being deﬁned as – “In formal education, a curriculum (plu-

ral curricula) is the set of courses, and their content, oﬀered

at a school or university”, while syllabus is deﬁned as “A

syllabus is an outline and summary of topics to be covered

in a course”. They have identiﬁed that in CBSE, the ad-

vantage is that the curriculum is same all over the country

and the continuity of education is not a problem if some-

one needs to change a school. They have also inferred that

ICSE syllabus is tougher than that of the CBSE and state

based school boards. Their research work showed that the

school boards are giving high importance to evaluation and

examination system which includes some additional forms

of evaluation such as (a) project work, (b) reading and writ-

ing skills, (c) participation in co-curricular activities, (d)

attitude and behaviours, etc. However major emphasis was

given on written examination by schools. IBO puts more

emphasis on project based and practical work compared to

the Indian school boards as it follows a global curriculum all

over the world. IBO assessment focuses on what skills the

students have learnt or what level of understanding can the

students demonstrate. British Council in India, in their re-

port [13] on the Indian school education system provides an

overall picture into this large and evolving school education

system of India. They remarked that “the present education

system in India is guided by diﬀerent objectives and goals

but is based around the policies of yester years.” They claim

that two important policies of the Government of India—the

Sarva Shiksha Abhiyan (SSA) in 2001 and the Right of Chil-

dren to Free and Compulsory Education (RTE) Act, 2009

have made education priorities rise among common people of

India and have been responsible for improvements in educa-

tional performance. However, this report does not mention

about the challenges faced by the Indian school education

system today. Also National Institute of Open Schooling

(NIOS) is not covered in these reports.

3. NEED AND SIGNIFICANCE OF THE STUDY

The objective of this research is to ﬁnd the trends and pat-

terns of cognitive level in secondary school board examina-

tion questions. To the best of our knowledge, there is no

previous research on a comparative analysis of the cognitive

level of questions from three diﬀerent school boards (with

respect to India) and in multiple subjects. This study tried

to address this gap through a comprehensive research and

provide an accessible and open dataset as a solution to the

problem and for future similar comparative research.

4. RESEARCH QUESTION

This study investigates the cognitive levels of the questions

used in the English medium school leaving examinations for

three national school boards in India administered nation-

wide from 2011 to 2020. It put forwards the following re-

search question: To what extent do Indian school board

school leaving examination questions cover the lower and

higher-order cognitive levels of Bloom’s Taxonomy?

5. DATASET DETAILS AND DESCRIPTION

5.1 Why are these data useful?

Every year, millions of students appear for the 10th board

(secondary exam) in India from various national and re-

gional state education school boards. In 2021, 21,50,608

students appeared in the 10th Board exam in Central Board

for Secondary Education Examination (CBSE) in India [1].

However, not much is studied about what Cognitive ar-

eas the board exams test, which sections of learning they

stress, and which ones they ignore. This research tries to

provide a comprehensive comparative analysis of three En-

glish medium national school boards – Indian Certiﬁcate

of Secondary Education (ICSE), the Central Board of Sec-

ondary Education (CBSE), and the National Institute of

Open Schooling (NIOS). The Indian Certiﬁcate of Secondary

Education popularly known as ICSE is an examination con-

ducted by the Council for the Indian School Certiﬁcate Ex-

amination (CISCE), a private board of school education in

India for Class 10. The CISCE board is headquartered in

New Delhi. The Central Board of Secondary Education

(CBSE) is a national-level board of education in India for

public and private schools, controlled and managed by the

Government of India. There are more than 27,000 schools

in India and 240 schools in 28 foreign countries aﬃliated

with the CBSE. The National Institute of Open Schooling

(NIOS), formerly National Open School was established by

the Ministry of Human Resource Development of the Gov-

ernment of India in 1989 to provide education to all seg-

ments of society with the motive to increase literacy and

aimed forward for ﬂexible learning. The NIOS is a national

board that administers examinations for Secondary and Se-

nior Secondary examinations similar to the CBSE and the

ICSE. NIOS enrolls about 350,000 students annually which

makes it one of the largest open schooling systems in the

world.

5.2 Who can beneﬁt from these data?

This data will help school education stakeholders to get an

insight into the cognitive level trend/patterns of assessment

Table 1: Dataset details

Sl.No. Board Years No. of Ques.

1 ICSE 2011-2020 838

2 CBSE 2011-2020 1274

3 NIOS 2011-2020 959

Total 3071

Figure 1: Types of Questions and Cognitive levels

questions.This data can be extremely useful and can be fur-

ther re-used for creating intelligent tools as mentioned in

similar research articles like [10] and [9]. The following ta-

ble, Table 1 shows the details of the dataset.

6. METHODOLOGY

The data was collected from the physical and digital copies

of the previous question papers. The text data of digital

copies were cleaned and curated in online spreadsheet sys-

tems (Google Sheets). Once compiled, all the questions were

segregated into individual sheets year and subject-wise

Three annotators having suﬃcient domain knowledge about

Cognitive levels using Bloom’s Taxonomy action verbs [11]

annotated each question individually to identify the most

appropriate cognitive level. The inter-annotator agreement

for the appropriate cognitive level was in the substantial

agreement range (using Fleiss’s kappa) [4]. For data analy-

sis and visualization, Tableau [15] was used.

7. RESULTS

7.1 General Findings

Table 2 and Table 3 shows the top ﬁve Bloom’s Taxonomy

action verbs used for Science and Mathematics across all

the three boards. Figure 1 shows the various types of ques-

tions, based on their cognitive level and their frequencies.

Figure 2 shows various types of questions and their frequen-

cies against each board. Figure 3 shows Cognitive level with

respect to individual boards.

7.2 Patterns and Observations

7.2.1 General Question patterns

Figure 4 (Mathematics), 5 (Physics), 6 (Chemistry), and 7

(Biology) shows the comparison of the number of questions

CBSE and NIOS takes a combined Physics, Chemistry and

Biology exam while ICSE have separate exams for the three

subjects.

Figure 2: Types of Questions and School Boards

Figure 3: A complete overview of School Boards, Question

typ e and Cognitive levels

Table 2: Top 5 Most used Action Verbs (Maths)

Rank Verb f CBSE f NIOS f ICSE

1 Find 565 220 192

2 Prove 96 36 25

3 Draw 36 9 19

4 Construct 32 18 14

5 Solve 32 11 19

f denotes frequency of verb in respective board.

Table 3: Top 5 Most used Action Verbs (Science)

Rank Verb f CBSE f NIOS f ICSE

1 Name 31 31 188

2 Write 68 45 96

3 Explain 44 63 92

4 State 47 32 92

5 Draw 44 30 94

f denotes frequency of verb in respective board.

Figure 4: Question Pattern - Mathematics

Figure 5: Question Pattern - Physics

asked on behalf of diﬀerent Cognitive levels of Bloom’s Tax-

onomy against each Board. Figure 8 shows an overall com-

parison of Boards, Question type and Cognitive levels. From

this analysis, it can be inferred which type of assessment

was most preferable for a speciﬁed cognitive level. As it

is observed, Application-level questions are most frequently

asked in all three types of questions (MCQ, Long, Short).

For the MCQ type of questions, there was no “create” and

“evaluate” type of questions asked. The reason can be that

MCQ questions are known as single marks questions, and

asking for creation and evaluation will be much more in

Figure 6: Question Pattern - Chemistry

Figure 7: Question Pattern - Biology

Figure 8: Question Pattern - Overall

terms of time and eﬀort required. In long and short type

questions, all Cognitive levels of Bloom’s Taxonomy ques-

tions can be found. It was also observed that lower cogni-

tive levels of Bloom’s Taxonomy questions are asked more

for the long answer types questions.

7.2.2 Board vs Subject vs Cognitive level

The highest number of application-level questions are asked

in Mathematics subject in all three boards. Physics also had

the highest frequency of Application level questions among

all three boards similar to Mathematics. For Biology sub-

ject most number of Knowledge level questions were asked

for CBSE and NIOS boards. At the same time, ICSE em-

phasises more on the understanding-type questions, as they

had the highest tally among all other cognitive levels. Sim-

ilarly for Chemistry, Knowledge level questions frequency

was highest among all three board subjects. For all three

boards for 9 years, there were no create level cognitive-type

questions asked in Chemistry.

7.3 Discussions

7.3.1 Analysis of the three boards for educational

and assessment practices

According to the study ﬁndings, lower order cognitive level

exam questions outnumber higher order cognitive level exam

questions by a wide margin. Such an imbalance in questions

based on the six cognitive domains in national examinations

may have a negative impact on instructional quality and stu-

dent learning. CBSE has two categories of papers in almost

every year, Delhi and Outside Delhi. These papers have

some diﬀerences in number of questions in each cognitive

level. In ICSE science, all questions have multiple subparts

(upto 5 or 6) each from diﬀerent chapters as well as cog-

nitive levels. These can be very confusing for the student

and can cause ambiguities in their analysis. Maths papers

throughout the three boards have an abundance of applica-

tion level questions. Most of these have the somewhat am-

biguous Bloom’s Taxonomy Action Verb ‘Find’. In NIOS

papers (Mathematics), the questions for visually impaired

students are usually of lower cognitive level than the equiv-

alent for the other students. NIOS Maths asks students to

name and deﬁne terms in the form of MCQs (approximately

5%) that comes under memorization level and is absent in

CBSE. CBSE papers before 2018 only test concepts from the

second half of the syllabus, as was the case in Continuous

And Comprehensive Evaluation (CCE) pattern.

7.3.2 The impact of secondary school national board

exams on teaching and learning quality

The questions indicated that students are required to spend

more instructional time preparing for the exam or studying

past exams that are heavily lower cognitive thinking (LOTS)

driven and derived from curriculum books. As a result, stu-

dents will fail to master complex reasoning skills as required

by the curriculum. Students are more likely to face chal-

lenges in secondary and tertiary education, as well as in

their personal and professional lives, if primary education

does not include critical thinking instruction and long-term

assessment.

8. LIMITATIONS OF THE STUDY AND FU-

TURE WORK

The study, however, was limited to the English medium na-

tional school board examinations. With 30 or more state

boards in regional languages, data collection and annota-

tion would have been a real challenge. However, if it had

been implemented, it would have provided a more in-depth

understanding of the national education system by reﬂecting

on assessment approaches.

9. CONCLUSION

All the Indian school boards are similar in terms of mak-

ing students remember the facts, understand the concepts

and apply them to solve problems. Thus the questions for

evaluation and assessment also focus on these three major

aspects. The CBSE syllabus is comparatively “more easy”

on students in its approach as it has been designed for a

speciﬁc year and is divided into various segments. Every

segment is given a speciﬁc number of periods so that it can

be completely and thoroughly taught in one year. It empha-

sises on understanding of concepts and processes with their

application. The ICSE system stresses more in terms of

aptitude development and thoroughness by almost equally

focusing the syllabus on remembering facts, understanding

of concepts and application of the processes learnt. NIOS

being an open schooling option, do provide some relaxation

in terms of cognitive rigour when compared to the other two

national boards. In conclusion we can say, if the syllabus is

written considering the Bloom’s Taxonomy and knowledge

dimension, it will be lot easier to analyze and evaluate that

whether the learning objectives have been successfully sat-

isﬁed on completion of the curriculum.

10. REFERENCES

[1] Cbse 10th results 2021.

https://www.ndtv.com/education/cbse-10th-result-

2021-9904-pass-result-under-process-for-over-16000-

students. [Accessed 14-Jan-2023].

[2] Rigor/Relevance Framework: A Guide to Focusing

Resources to Increase Student Performance —

leadered.com. https://leadered.com/papers/rigor-

relevance-framework-a-guide-to-focusing-resources-to-

increase-student-performance/. [Accessed

20-Jan-2023].

[3] L. W. Anderson. Objectives, evaluation, and the

improvement of education. Studies in educational

evaluation, 31(2-3):102–113, 2005.

[4] R. Artstein. Inter-annotator agreement. In Handbook

of linguistic annotation, pages 297–313. Springer,

2017.

[5] I. R. Assaly and O. M. Smadi. Using bloom’s

taxonomy to evaluate the cognitive levels of master

class textbook’s questions. English Language Teaching,

8(5):100–110, 2015.

[6] S. Chadwick. Curriculum development in orthodontic

specialist registrar training: Can orthodontics achieve

constructive alignment? Journal of orthodontics,

31(3):267–274, 2004.

[7] S. Das. Cognitive Level Analysis in a Learning Cycle.

In 2018 IEEE 18th International Conference on

Advanced Learning Technologies (ICALT), pages

449–451, Mumbai, July 2018. IEEE.

[8] S. Das and A. Basu. A sample study on the

distribution of indian school curricula over bloom’s

cognitive domain categories. In EDULEARN16

Proceedings, 8th International Conference on

Education and New Learning Technologies, pages

4811–4817. IATED, 4-6 July, 2016 2016.

[9] S. Das and A. Basu. A sample study on the

distribution of indian school curricula over bloom’s

cognitive domain categories. In EDULEARN16

Proceedings, pages 4811–4817. IATED, 2016.

[10] S. Das, S. K. Das Mandal, and A. Basu. Cognitive

complexity analysis of learning-related texts: A case

study on school textbooks. In International

Conference in Methodologies and intelligent Systems

for Techhnology Enhanced Learning, pages 74–84.

Springer, 2020.

[11] S. Das, S. K. Das Mandal, and A. Basu. Classiﬁcation

of action verbs of bloom’s taxonomy cognitive domain:

An empirical study. Journal of Education,

202(4):554–566, 2022.

[12] N. M. Freahat and O. M. Smadi. Lower-order and

higher-order reading questions in secondary and

university level eﬂ textbooks in jordan. Theory &

Practice in Language Studies, 4(9), 2014.

[13] B. C. India. Indian school education system-an

overview. retrieved march 20, 2017, 2014.

[14] D. R. Krathwohl. A revision of bloom’s taxonomy: An

overview. Theory into practice, 41(4):212–218, 2002.

[15] A. Ohmann and M. Floyd. Creating Data Stories with

Tableau Public. Packt Publishing Ltd, 2015.

[16] T. V. Ramirez. On pedagogy of personality

assessment: Application of bloom’s taxonomy of

educational objectives. Journal of personality

assessment, 99(2):146–152, 2017.

[17] W. Sundayana, P. Meekaeo, P. Purnawarman, and

D. Sukyadi. Washback of english national exams at

ninth-grade level in thailand and indonesia. Indonesian

Journal of applied linguistics, 8(1):167–176, 2018.