Annual Report on Assessment at ESU

Assessment and Teaching Enhancement Center

July 2000

ESU's assessment program as it has evolved to the present can be characterized by the following principles:

1) There are institution-wide commitments to attempt to understand what takes place when students learn and to improve the learning experience for students at ESU.

2) It is recognized that different approaches to assessment are appropriate at different levels and in different settings. Thus, a department chair⁽¹⁾ may personally interview graduating seniors about their academic experience while the central administration conducts a telephone survey of a sample of the entire student body to ask about instructional effectiveness. Even in the same academic unit, different assessment techniques may be used for different programs.

3) The success of the assessment program is indicated by changes which attempt to improve student learning as well as by changes which can demonstrate such improvement. Experimentation and special study do not always produce solutions for problems, but they do signify a willingness to seek solutions.

Thus, assessment at ESU is diverse and decentralized. The assessment of majors is the responsibility of the faculty in the various departments. Multiple measures of achievement are employed in all academic programs, but there is no mandated set of measures for all majors.

This document updates the status of assessment at Emporia State University, largely based on the annual reports submitted by the department chairs. The individual reports can be obtained from the ESU network at h:/vpaa/common/assess_reports/.

Changes in Assessment

Table 1 updates the data from previous years about the assessment techniques used in the various academic programs. It shows the continuance of some trends, but some new directions as well. Student and alumni surveys (or focus groups) continue to be used in the overwhelming majority of programs. There is a decline in the use of persistence studies. Most significantly, different techniques have been developed and are being used across the campus. Most programs now employ capstone experiences, writing skills assessment, and data concerning the placement of their graduates. Many programs use exit interviews, employer surveys, and some form of portfolio analysis in their program assessment.

Table 1
Summary of Assessment Techniques Employed

Assessment Activity	1988 (n = 65)⁽²⁾		1998 (n = 58)²		2000 (n = 50)²
Assessment Activity	Majors	%	Majors	%	Majors	%
Locally Developed Achievement Measures	16	24%	40	69%	24	48%
External Expert	23	35%	23	40%	21	42%
Self-Reported Data Collection⁽³⁾	39	59%	NA	NA	NA	NA
Student Surveys (or Focus Groups)³	NA	NA	41	71%	33	66%
Alumni Surveys (or Focus Groups)³	NA	NA	42	72%	35	70%
Nationally Standardized Achievement Tests	12	18%	11	19%	13	26%
Persistence Studies	13	20%	14	24%	6	12%
Portfolio Analysis	NA	0%	19	33%	20	40%
Capstone Experience⁽⁴⁾	NA	NA	34	59%	35	70%
Writing Skill Assessment⁴	NA	NA	37	64%	29	58%
Exit Interviews⁴	NA	NA	27	47%	20	40%
Placement of Graduates⁴	NA	NA	43	74%	33	66%
Employer Satisfaction Surveys⁴	NA	NA	26	45%	20	40%

Tables 2 and 3 show the growth in the use of multiple assessment measures in the various programs. Nearly all of ESU's undergraduate and graduate majors are evaluated by multiple measures. Most of the undergraduate majors and almost that proportion of graduate majors employ six or more measurement techniques.

Table 2

Growth in Use of Multiple Measures in Undergraduate Programs: 1988-2000

Number of Measures	1988	1991	1994	1998	2000
0 Types of Assessment	33%	0%	0%	0%	0%
1 Type of Assessment	18%	2%	7%	5%	7%
2 Types of Assessment	22%	30%	13%	5%	0%
3 Types of Assessment	16%	25%	28%	5%	13%
4 Types of Assessment	10%	28%	28%	17%	3%
5 Types of Assessment	0%	10%	21%	20%	17%
6 or More Types of Assessment	0%	5%	3%	54%	60%

Table 3

Growth in Use of Multiple Measures in Graduate Programs: 1994-2000

Number of Measures	1994	1998	2000
0 Types of Assessment	0%	0%	0%
1 Type of Assessment	0%	0%	0%
2 Types of Assessment	0%	6%	15%
3 Types of Assessment	14%	24%	10%
4 Types of Assessment	55%	0%	10%
5 Types of Assessment	27%	24%	20%
6 or More Types of Assessment	5%	47%	45%

Appendixes A and B list the assessment techniques used in each undergraduate and graduate major.

Actions Taken at the Program Level as a Result of Assessment

Assessment is an ongoing process that has as its goal improved student learning. If assessment is to be successful, it must provide the basis for change. The individual departmental reports document many changes which have been made in the last year to improve student learning. Following is a summary that illustrates the range and diversity of actions taken.

Development of internships, clinical practice, and other experiential learning experiences in Psychology and Nursing.

Installation of smart classrooms and enhanced integration of computer technology in all Business programs and Communication and Theatre Arts.

Modification of major requirements and course descriptions in most departments and schools; special attention to curricular reform to meet accreditation requirements in Art, the School of Business, Library and Information Management, and Physical Sciences; development of new core curricula in Biological Sciences, Communication and Theatre Arts, School Leadership, and Nursing.

Course redesign and other curricular change to place renewed emphasis on critical thinking in Biological Sciences, Mathematics and Computer Science, Nursing, and Sociology and Anthropology.

Integration of hands-on computer experiences in classes in some Business programs and Library and Information Management.

Increased emphasis on learning to work in teams and the development of interpersonal skills in Business Administration.

Moving from viewing global issues as a separate topic to integrating these issues into existing classes in Business Administration.

Development of discipline-specific freshman seminar classes in Biological Sciences, Communication and Theatre Arts, and Music.

Revision of position descriptions to increase faculty expertise on multicultural and diversity issues in Social Sciences.

A paleontology minor and concentration for the Earth Sciences major were developed and forwarded to the Board of Regents for approval.

A new undergraduate major in Information Resource Studies was developed in Library and Information Management with help from faculty in Business and Liberal Arts and Sciences.

Increased web support for campus-based classes in Physical Sciences, Communication and Theatre Arts, and Psychology and Special Education.

Development of online classes to serve place-bound students in Communication and Psychology.

Development of centralized advising in all Business programs.

Organization of remote advising services for students in the regional programs in Library and Information Management.

Actions Taken to Improve Assessment

In addition to changes to improve student learning, the assessment process itself must be assessed from the viewpoint that it can always be improved. Following are some actions taken last year to improve assessment at the program level.

Formation of advisory councils (consisting of alumni, employers, and other external parties) at the departmental level to review assessment data and make appropriate recommendations in all Business programs.

Formation of a student advisory board to participate in program assessment in Foreign Languages; a strengthened role for student organizations in Physical Sciences.

Development of instruments to assess online classes in the Teachers College and the School of Library and Information Management; development of instruments to assess online programs in the College of Liberal Arts and Sciences.

Development of a form to assist faculty in identifying and remedying individual student weaknesses in Music Education.

Campus-wide Activities

Several campus-wide assessment projects either are ongoing or were completed during 1999-2000. Each of these activities has a separate report available which is not included in this document.

The Regents Student Perceptions Study is a telephone survey of attitudes of undergraduate students toward their educational experience at ESU. The report will be presented to the Kansas Board of Regents this fall. An Assessment of Undergraduate Advising was also completed this year, and the report is currently being disseminated among the academic units. An Assessment of Developmental Courses was conducted during the Fall semester of 1999 at the request of the Academic Affairs Committee of the Faculty Senate. An Assessment of the BIS degree (the online version for place bound students) was completed during December of 1999 and is available from the Assessment and Teaching Enhancement Center. The General Education Council has nearly completed a Comprehensive Assessment of General Education. The Teachers College with the assistance of College of Liberal Arts and Sciences in involved in a project to develop measures for Multicultural/Diversity Outcomes Assessment. The annual Employment/Enrollment Report is available from the Office of Career Services.

Observations and Recommendations

The faculty in most of the academic programs have adopted assessment as an important part of their efforts to improve the environment for learning. Central and school/college administrative support for assessment remains strong. Some commendable practices are evident from the 2000 reports. Here are but a few that seem noteworthy.

The School of Business's Annual Assessment Forum brings faculty and administrators together for a frank and honest discussion of assessment data and the issues these data suggest. This is definitely not an exercise in public relations, and such candid discussions may pose some risks. Nevertheless, these forums suggest a clear commitment toward establishing a "culture of evidence" in the school that will pay dividends during the next accreditation visit.

The efforts to develop improved ways to assess online instruction in the School of Library and Information Management and the Teachers College indicate that a rigorous assessment can be compatible with the requirements to reward individual faculty performance. There is much to be done in this area of assessment, but clearly these two units are displaying leadership that will be recognized beyond the ESU campus.

The comprehensive approach to the assessment of a performing arts major conducted by the faculty in the Theatre program is impressive in both what it involves and what it has accomplished. This is the only program at ESU that requires students to be tested over the content of the major as entering majors and as seniors. These test data together with exit interviews and external expert evaluations of the majors' performance and technical skills give the faculty clear evidence about what these students learn from their curricular and co-curricular experiences.

Possibly no program at ESU comes closer to establishing a culture of evidence than the Department of Physical Sciences achieves in its multi-methodological approach to assessment. The formal plan includes six strategies, and several other assessment techniques are used as well. The result is a practical implementation of the philosophy of assessment that provides a model for what the North Central Association has identified as the third (or highest) level of implementation.

While the work is not complete, the efforts to assess and improve critical thinking in Biological Sciences, Mathematics and Computer Science, Nursing, and the General Education Council deal with perhaps the single most important issue in student learning that postsecondary educators face.

A final project not yet complete is the Multicultural/Diversity Outcomes Assessment underway as a collaborative enterprise of the Teachers College and the College of Liberal Arts and Sciences. Supported by a major Department of Education grant, this project will have national impact on the way that students are assessed on these critical social skills.

Assessment at ESU is working well. Nevertheless, there are always some improvements that can be made. Following are some actions and recommendations that emanate from experience of reviewing the individual program reports.

1. The original assessment plan for ESU was developed with an emphasis on improving undergraduate education. While we are using assessment techniques in most of our graduate programs, our sophistication and comfort with assessment at the graduate level need to be developed. Appendix C contains a copy of a memo proposing an initiative on the Assessment of Graduate Programs at ESU.

2. ESU's leadership in developing methods for assessing online instruction is acknowledged above. Still, these instruments are new and need to be tested. There are many other issues that relate to the problem on insuring student learning in distance classes. All academic units who are offering courses and programs via distance learning technologies should give special attention to assessment issues. If existing assessment methodologies are not adequate or appropriate for these programs, then new types of assessment should be developed.

3. Our approach is highly decentralized, and there are clear benefits to "local control." Yet we need to do more to share assessment practices and results on a campus-wide basis. We should also emphasize the relationship of assessment to all the other activities in which we are engaged. ESU should begin this year to sponsor an Annual Forum on Assessment in which appropriate issues are discussed and improvements in practices are explained.

4. The North Central Association modified its criteria for accreditation in March of this year. The prior emphasis on developing and implementing an assessment "plan" has been replaced with a requirement that post-secondary institutions work toward establishing an assessment "culture." We need to disseminate widely the new North Central policy statements and revise our plans accordingly. The March 2000 Addendum to the Handbook on Accreditation may be obtained from http://www.emporia.edu/asem/handbook_supplement.pdf or the ESU network at h:/vpaa/common/assess_reports/Handbook_Supplement.pdf (requires Adobe Acrobat Reader).

5. We currently have several committees charged with responsibilities for monitoring assessment at the university level (e.g., the General Education Council and the Task Force on Assessment) and "local" committees in about half the departments. The new North Central requirements call for a more formal governance structure for dealing with assessment. The current Task Force on Assessment should become a standing Committee on the Assessment of Undergraduate Programs appointed by the President. The Graduate School should establish a standing Committee on the Assessment of Graduate Programs also appointed by the President. Each school or college should establish policy concerning departmental assessment committees.

Submitted by:

Edward L. McGlone

Director of the Assessment & Teaching Enhancement Center

July 2000

Appendixes A & B

Abbreviations Used

Assessment Techniques

LD AC Locally Developed Achievement Measures

EX EV External Expert Evaluation

NA AC Nationally Standardized Achievement Tests

PE Persistence Studies

PO Portfolio Analysis

CA Capstone Experience

WR Writing Skill Assessment

EX Exit Interviews

ST SU Student Surveys (or Focus Groups)

AL SU Alumni Surveys (or Focus Groups)

PL Placement of Graduates

EM Employer Satisfaction Surveys

Graduate Majors

ACC Accounting

ARTT Art Therapy

BA Business Administration

BE Business Education

BIO Biology

CE Counselor Education

EA Educational Administration

EC Early Childhood

ENG English

IDT Instructional Design

LIM Library Science (MLS)

LID Library & Information Management (PhD)

MA Mathematics

MT Master Teacher

MUS Music

PE Physical Education

PS Physical Sciences

PSY Psychology

SE Special Education

TE Teacher Education

Undergraduate Majors

ACC Accounting

ART Art

ART BSE Art Education

BE Business Education

BIO Biological Sciences

CIS Computer Information Systems

CO Communication

CO BSE Communication Eucation

CS Computer Science

ECON Economics

ELED Elementary Education

ENG English

FL Foreign Languages

FL BSE Foreign Languages Education

FL ESL English as a Second Language

MA Mathematics

MA BSE Mathematics Education

MMF Business Administration, Management, Marketing & Finance

MUS Music

NUR Nursing

OS Office Systems Management

PE Physical Education

PS Chemistry, Earth Science

& Physics

PS BSE Physical Sciences Education

PSY Psychology

REC Recreation

REH Rehabilitation

SS Social Sciences

SOC Sociology and Anthropology

TH Theatre

Appendix A

Assessment Techniques Used in Undergraduate Programs at ESU

ACC

ART

ART BSE

BIO

CIS

CO BSE

ECON

ELED

ENG

FL BSE

FL ESL

MA BSE

MMF

MUS

NUR

PS BSE

PSY

REC

REH

SOC

Appendix B

Assessment Techniques Used in Graduate Programs at ESU

ACC

ARTT

BIO

ENG

IDT

LIM

LID

MUS

PSY

Appendix C

Memo Proposing Initiative on Assessment of Graduate Programs

To: Vice President John Schwenn and Dean Tim Downs

From: Ed McGlone

Subject: Assessment of Graduate Programs

Date: May 31, 2000

I write to recommend that a high priority be assigned to the assessment of graduate programs at ESU during the coming academic year. I am also proposing some of the activities needed to achieve this priority.

I make this recommendation for several reasons. While we have been commended and awarded for our efforts in assessment at the undergraduate level, we have not placed any special emphasis on assessment at the graduate level. A review of the reports submitted by the departments this year reveals an inconsistency in both the understanding and the accomplishment of assessment at the graduate level. I am also concerned about the coming North Central study and visit which will be completed during the 2003-2004 academic year. I am not sure that we can receive unconditional re-accreditation unless we move strongly in the area of implementing assessment activities in all academic programs-graduate as well as undergraduate.

You may be aware that the latest Addendum to the Handbook of Accreditation, Second Edition (North Central Association, March 2000 indicates that full compliance ("level three" in North Central terminology) requires that the graduate as well as undergraduate programs at an institution have assessment programs which are reviewed and updated annually. Failure to demonstrate that "assessment has become an institutional priority, a way of life" is grounds for an North Central Evaluation team to call for a monitoring report (re-review after three years) or a progress report (re-review after one year).

Therefore I propose the following:

1. ESU should conduct an assessment of Graduate Student Perceptions modeled on the Regents Student Perception Survey for undergraduates. I further suggest that we conduct this survey on a biennial basis alternating years with the undergraduate survey.

2. ESU should conduct an assessment of graduate advising, again somewhat patterned on this year's study of undergraduate advising. The purpose here will be to provide baseline data, and the task of conducting subsequent annual evaluations of graduate advisement should be assigned to the departments.

3. The departments that offer graduate degrees should develop formal Assessment Plans, much as we did for the undergraduate programs nearly twelve years ago. This time we need to be guided by the new reference materials in Chapter 4 (Criteria) of the North Central guidelines, "Assessment of Student Achievement: Levels of Implementation."

4. I also recommend that the Graduate School establish a standing Committee on the Assessment of Graduate Programs. This might be a subcommittee of the Graduate Council or it could be a freestanding entity. Its purposes would be to review and approve the departmental plans and to advise the Assessment Director in the conduct of the student perceptions, advising, and other appropriate studies. (This is also pretty much mandated by the new North Central guidelines). This committee should be appointed by and answer to the Graduate Dean.

We should be confident that we can achieve a high level of accomplishment in assessing graduate programs. Assessment is part of the ESU culture. But we will need the commitment and involvement of all the deans, as well as the chairs of the departments offering graduate degrees. I firmly believe that we need an institutional mandate now to start us moving in the right directions.
I hope that these issues will be brought to the attention of the Deans' Council, and I ask that Vice President Schwenn provide a formal response to this proposal. Thanks!

1. In June of 2000, the term "department" replaced "division" in reference to the academic units. Also, some reorganization of academic programs occurred. Where possible, this document reflects the new terminology and reorganization.

2. Differing ways to count majors have been used, but the percentages are not affected substantially.

3. Self-Reported Data Collection" in the 1988 report included both Student Surveys and Alumni Surveys.

4. Technique not included in the 1988 document

Last Updated March 19, 2007

Assessment & Teaching Enhancement Center

ESU Quicklinks

Assessment & Teaching Enhancement Center Links

Annual Report on Assessment at ESU