Psychology Department Assessment Report

2000
 
 
 
 
 

Summary






This year’s assessment of the Psychology Department includes the following:

· An elaboration on our Goals to include Performance Criteria.

· A faculty survey to determine whether the courses that are offered include enough assignments to allow the students to meet these performance criteria.

· Further analysis of last year’s student survey, and the implementation of an additional student survey designed to assess the effectiveness of various types of capstone projects.

· Further administration of the pre-test and post-test to our Gen. Ed. classes.





1. Changes made to the program as a result of last year’s assessment.

A. Revision of Psyc 200 (Survey of Psychology).
  Psyc 200 is the introductory course that is required of all Psychology majors. It is taught using the Personalized System of Instruction format, which means that students read the text at their own pace and take quizzes to show that they have mastered the material. Starting Fall semester, 2000, major changes have been made in the format of the course. These changes are meant in increase student engagement in the course by making it more relevant to their lives. In addition, these changes are meant to better address our Goals B, C, and D (see Appendix A).
  1. We have added learning activities to supplement the multiple-choice quizzes. Students will be required to complete four activities. Each activity involves going to at least one Web site, reading some material, carrying out an interactive on-line activity, and then answering a series of short-answer and/or short essay questions. The performance of these activities will introduce students to Goals B (understanding scientific method), C (written communication), and D (use of computers).

2. Another change is that from now on, faculty will more closely monitor students' progress in taking the quizzes and completing the learning activities. If a student falls behind, a faculty member will try to contact them to meet with them and help them catch up.

 After two semesters of the revised course, we are going to assess its effectiveness by comparing grades received, and drop-out rates, from this year to last year.
 
  B. Improvement of student advising.
  One of the things we learned from last year’s student survey was that many students felt that they were not fully informed as to what the requirements are that must be fulfilled for graduation. We have made two changes in our program to rectify this situation.

1. New Major Orientation.
 

We have always held an orientation session in October for new majors. At this session we have gone over all of the Department's requirements. Last year we asked those students who attended to fill out a survey, and they all reported that they found this orientation extremely helpful. Therefore, we will now require all students who have declared a major in psychology since last September to attend this orientation. 2. Advising
  Last year's student survey showed that one-third of our students had never been to see their faculty advisor. Many of our students never see their advisor and then, when they apply for graduation, are surprised to learn that they have not met all of the requirements. Therefore, starting this year, all students will be required to meet with their advisors at least twice; once in the first term in which they declare their major, and once when applying for graduation. Registration holds will be placed on students until they have met with their advisor. 3. Graduate school workshops
  Many of our students have expressed confusion about the process of applying to graduate school. To address this, Dr. Suzanne Gaskins has started a "graduate school application workshop" for seniors who are thinking about applying to graduate school. They meet once every two weeks to discuss topics such as GREs, choosing a school, filling out the application etc. We also hope that this will create more cohesion amonst our students by giving them a chance to meet with other students with similar interests.


  2. Goals and performance criteria. Last year we decided on four Goals that students majoring in Psychology should meet. This year we elaborated on those goals and added specific Performance Criteria. (See Appendix A).

Goal A (Dissemination of content) will be met in all Psyc courses, but especially in Psyc 200 and our required Core courses.

Goal B (Understanding of scientific method) will also be met in all Psyc courses, but especially in our required series of Statistics and Research Methods I and II (Psyc 202 and 302), and Lab courses.

Goal C (Written and oral communication) is met in most of our Core courses, and by completion of the Capstone project.

Goal D (Computer literacy) is met by the new Psyc 200 (see above), Statistics and Research Methods II (Psych 302), and the lab courses.

3. Faculty survey After elaborating our Goals and Performance Criteria, the first thing we thought it was important to do was to determine whether students are being given adequate opportunity to satisfy them. In other words, will completion of the Psyc major requirements expose them to each of these goals? To determine this, all Psyc faculty filled out a survey for each of the courses they had taught in the last three years (see Appendix B). Since we felt quite confident that Goals A and B were being met in a number of courses, this survey focused mainly on Goals C and D (writing, oral presentation, and use of computers). Following are the results of the survey, broken down by the goals and by the different types of courses. Psyc 200 (General), was not included in this survey because its requirements have changed substantially since the last time it was taught.

Written Communication

The survey asked the number and length of papers required, whether revisions were required, and the criteria used in scoring them. Questions were asked about other written assignments (e.g. homework), and essay questions on exams.
 

Psyc 202 and 302 (Statistics and Research Methods I and II). These are the only two courses, beyond Psyc 200, that all of our majors are required to take. Psyc 202 is a prerequisite for most of our 300-level courses, and Psyc 302 is an additional prerequisite for all lab and Capstone courses. Psyc 202 requires three short (2 page) papers, and Psyc 302 requires one short (2 page) and one longer (7 page) paper. Both of these courses require revision of the paper. Both include APA style, spelling and grammar, references, overall organization, and content in their scoring criteria.

Core courses. These are 300-level courses in three different categories. Each student must take at least one course from each of the three categories. Of the eleven courses covered in the survey, ten included substantial writing requirements. All of these required at least one long (about 10 pages), or multiple shorter (usually about 5 pages) papers. All of them used all of the criteria listed above in grading. Five of these ten required that students revise their paper at least once. In addition, eight of these courses required some other type of written assignment, such as homework, and essay questions on the exams.

The only core course that does not include much writing is Physiological Psych., for which it is felt that writing is less appropriate at the undergraduate level. That course does include occasional written homework assignments.

Lab courses. All students are required to take one lab course. All of the five courses covered in the survey required at least one major (10 or more pages) and / or multiple shorter (about 5 pages) papers. All included revisions, and all used the criteria listed above in grading. Three of the five also included other written homework assignments.

Elective courses. These are courses that are not required of students but may be included in the total number of psychology credit hours required. Of the eleven of these courses included in the survey, six could also be used to fulfil students' Capstone requirement (see below). All of these courses (whether or not they were a Capstone) required at least one major (10 or more pages) and / or multiple shorter (about 3 pages) papers. Most of them ask for revisions, and some ask for multiple revisions. All but one include additional written homework assignments, and all but one include essay questions on the exams.

Capstone. All students are required to fulfil the Capstone requirement either through Independent Study / Research, Field Experience, or by taking a class that has been designated as a Capstone class. Regardless of which class a students uses to fulfil this requirement, they must write one paper that is at least 15 pages long and includes at least 15 references. These papers are scored on all of the criteria list above. (For a more detailed description of the scoring criteria, see our 1999 Assessment Report, Appendix B).

A few faculty said that they would like to require more writing, but cited lack of time as the reason they didn't.
 
 

 Oral Communication
  The survey asked faculty whether students were required to give any kind of oral presentation, how long it was supposed to be, and whether it was graded. Neither Psyc 202 nor 302 includes an oral presentation. Of the eleven core courses surveyed, six did require students to give at least one short oral presentation. All but one of these graded the presentation. Four of the five labs included in the survey required students to give at least one oral presentation lasting at least 15 minutes. All but one of these graded the presentation. Five of the six elective courses that could count as Capstone required that students give at least one oral presentation that was graded. Only one of the five non-capstone electives required an oral presentation.

Several faculty reported that they would like to include more oral presentations but cited lack of time as the reason they didn't. Most of the courses that did include an oral presentation had a relatively small class size (25 or fewer).
 
 

 Computer Literacy
  The survey included questions about whether the class was held in one of the TEC rooms, the amount of time computers were used in class, and the extent to which computer use was required for out-of-class assignments.

Psyc 202 and 302. Psyc 202 is not held in a TEC room, but does require students to use computers for word processing and library searches (Psyc Info). Students in 202 are encouraged, but not required, to use e-mail and the Web. Psyc 302 is taught in a TEC room and includes approximately 1 hour per week working on the computers in class, using SPSS for Windows (a statistics software package). This class would include even more in-class experience with SPSS if it weren't for the fact that the version owned by the University does not have the capacity to do some of the statistics students are required to learn. Students in this class are also required to use computers outside of class for statistics, word processing, and obtaining assignments on the Web. They are not required, but are encouraged, to use e-mail and Psyc Info.

Core courses. None of the core courses are taught in the TEC rooms, but they all require computer usage at least for word processing. In addition, eight out of the eleven require the students to use Psyc Info, or some similar software, to search for references. All eleven of the core courses also encourage computer use for at least one other topic, such as e-mail or using the web. Four of eleven courses had included in-class demonstrations using the university's multi-media cart.

Lab courses. Four of the five lab courses are taught in a TEC room. These four courses require students to use the computers in class for between 20% and 50% of all class time. The main in-class use of computers is for statistics, although at least one lab class uses a separate software package for the collection of data. The one lab that is not taught in a TEC room requires students to use statistics outside of class. In addition to statistics, all five of these courses either require or encourage students to use computers for word processing, searching the web, and searching for references. Three of the five also encourage use of e-mail, and one requires the use of spreadsheet software.

Electives. None of the elective courses is held in a TEC room. Only one had ever used the University's multimedia cart, and that class used it only once. However, all eleven of the elective courses required that students use computers out of class for word processing, and all but one required the use of Psyc Info or some other library search software. Two required use of the web, and the other nine encouraged it. In addition, one required computer use for statistics and two required e-mail.
 

 Most faculty in all categories reported that they would like to use computers more in their classes. The main reasons for not doing so were lack of adequate facilities or software, and / or lack of knowledge of how to use them.

Summary of faculty survey.

Overall, the results of the faculty survey show that our courses go a long way toward providing students with adequate learning experiences to meet our goals. We appear to have made the most progress in writing. A student cannot graduate with a degree in Psychology without having written several papers, homework assignments, and essays on exams.

Although fewer courses involve an oral presentation, many of them do and the majority of our students can be expected to have given at least one oral presentation before an audience by the time they have graduated. In addition, a large number of our students present at NEIUs annual research and creative activities symposium.

The main reason faculty do not require more oral presentations seems to be class size. Many faculty said that there was simply not enough time during the semester to require all students (usually about 35) to give their own oral presentation.

For use of computers, the situation is also very good. Extensive computer usage is required in two courses that all students must take: Psyc 302, and the Lab. Any student who graduates with a degree in Psychology will have used computers extensively for word processing, statistics, and searching for library references. They will most likely have used them for e-mail and the web as well. Once the new requirements for 200 are in place, all will have been required to use the web.

However, most of our faculty feel that they would like to involve computers in their classes even more. Most of them reported that either the university's facilities, or their own knowledge, was inadequate to give the students as much computer knowledge as they would like. Possible solutions to this are discussed below, under #6.

4. Capstone analyses All of our majors are required to complete a Capstone project. This requirement can be met in one of three ways: Independent Study research project, Field Experience, or an enhanced upper-level course. More students tend to choose to an upper-level course as their Capstone.

All Capstones involve writing a final paper that is at least 15 pages long and includes at least 15 references. Most students complete this project during their senior year. We wished to determine whether students were actually getting what we expected them to get out of the project, and whether there were any differences between the three different methods of meeting the requirement. We addressed these questions in two ways: through a Capstone survey given to students who were about to graduate, and through analysis of the final papers the students turned in.

A. Capstone survey
 

A total of 23 students who were about to graduate were surveyed in Fall of 1999 and Spring of 2000. Fifteen of them had used a class to fulfil their capstone requirement, 4 had used Independent Study / Research, and 4 had used Field Experience. A copy of the survey administered can be found in Appendix C. Figures 1 through 4 show the results for the three different types of capstones for questions regarding how much they learned from their capstone, how well they felt it prepared them for the future, and their overall satisfaction with NEIU and the Psychology Dept. All of these questions were rated on a Likert scale with 1 indicating "not at all", and 5 indicating "a great deal".


 

The Figures show that for most of the questions, students who did an independent research project for their Capstone were the most satisfied, followed by students who did Field Experience. The only exception to this was the question about how well the Capstone project prepared them for what they wanted to do in the future, for which students doing Field Experience gave the highest ranking. Due to the low numbers, the only one of these to reach a conventional level of statistical significance was the question regarding satisfaction with the Psychology Dept.
 

 B. Analysis of final papers.
  The analysis of students’ final Capstone papers is in the preliminary stage. For the last two years we have asked all faculty who supervise Capstone projects to turn in a copy of the final papers, including the grade received, to the Psychology Dept. Unfortunately, compliance with this request has not been 100%. Eventually we would like to examine a number of these final projects in detail, to determine whether the performance criteria under Goal C are being satisfied.

Of the 14 papers for which grades were available, 6 were written by students who used a class for their capstone project, 3 were from a Field Experience, and 5 were from an Independent Research project. The grades students received on their papers are shown in Fig. 5.


 

Overall, the grades received were quite high (M = 3.36 on a 4 point scale). This is to be expected, since most faculty give students one or two chances to rewrite their papers. Independent research projects received the highest grades, followed by Field Experience.
 
 

 C. Discussion
  Although these results are preliminary, and more data need to be collected, both the grades received and the results of the student survey suggest that students who do an Independent research project learn more, and are generally more satisfied with their education. These students are closely followed by those doing Field Experience. This suggests that, in the future, we should encourage more students to consider doing an Independent Study or Field Experience instead of a class. There is no evidence that the enhanced class option is any easier than the other two.

Of course, it is possible that the students who are more satisfied to begin with are the ones who choose to do Independent study. In the future, this possibility might be tested by giving them the survey before and after doing the Capstone.

5. General Education

Our two General Education courses, Intro. to Psych. (Psyc 100) and Life Span Development (Psyc 110) were assessed separately in the same way they have been in previous years.

A. Psyc. 100 We again administered a pre-test and post-test to several Psyc. 100 classes in Fall of 1999 (N = 38 students) and Spring of 2000 (N = 56 students). (See previous assessment report for a description of the test). As in years past, there was a significant increase from the pre-test to the post-test for the Intro questions, which are designed to measure what students are supposed to learn in that class (see Fig. 6). There was not a significant increase in the Life Span questions, which are included as a control for practice effects and are not expected to increase.


 
 

This was the first year in which the test was administered to an Honors section of the course. Although the Honors sections scored substantially higher (posttest M = 60%) than the other sections (M = 42%), the difference was not statistically significant because there were only 7 students in the Honors section.
 

 B. Psyc 110. This year Psyc 110 was involved in a very detailed assessment of writing skills conducted by Dr. Margaret Condon. It included essays written by students at the beginning and end of the semester, and writing apprehension questionnaires filled out at various points during the semester. These data are still undergoing analysis and will be reported at a later date.

6. Changes to program recommended as a result of this assessment. A. Addition of more computer resources

The results of the Faculty Survey suggest that faculty are not using as much technology in their classes as they'd like for two main reasons: Lack of university facilities, and lack of their own ability. There are several things we plan to do in the coming year to address these issues.

First, we plan to obtain more software that can be used in classes. We have already purchased a version of SPSS for the Department that includes the Advanced Statistics module, which is necessary for most research that our students will be analyzing. However, since we have only one copy of it, it can be used by our students conducting research as part of an Independent Study, but it cannot be used for a whole class. Therefore, we are going to try to persuade the university to install it on the computers in at least one TEC room. This could be used by all students in Psyc 302, and by most of the labs.

The Department also recently purchased some software that can be used to present stimuli and obtain data in psychology studies. This software could be very useful in a lab class, but, again, we don't have enough copies for a whole class. Therefore, we will look into smaller, relatively inexpensive versions of this type of software that can be purchased by each student individually.

Finally, there is the problem of faculty not feeling comfortable using the facilities that do exist. Particularly striking is the fact that almost none of the faculty reported using the university's mulit-media cart, which is available for use in any class. The main problem is that most of us don't know how to use it, and it is not easily available for us to practice using outside of class. Furthermore, some of the applications we would like to use have not been installed on it. Therefore, this year our Department bought our own overhead projection system that can be used for the same purpose. This system will be connected to one of our computers, on which we can install any software we want. In addition, faculty will be able to bring in their own laptops and connect them to the system. Our hope is that all faculty will learn how to use this system, and will therefore increase their use of technology in the classroom.

B. Increasing oral communication skills

The results of the faculty survey suggest that students may not be getting enough opportunities to enhance their oral communication skills. This problem is more difficulty to solve, because the main reason is lack of time. We plan to discuss this problem in our faculty meetings this year. Possible solutions might be requiring that all lab courses include an oral presentation, or requiring some type of presentation outside of class.

C. Encouragement of more research and field experience.

Based in part upon the results of the Capstone survey, we would like to encourage more students to do either an Independent Study, or Field Experience, for their Capstone, instead of a class. One way we can encourage this is simply by letting them know the results of this survey, and the grades given on final papers. We will do this by presenting the results at our annual orientation for Psychology majors, which is attended by a large number of students shortly after they declare their major. We will also put the results on our Web page, so they will be seen by students who were not able to attend the orientation. In addition, several students indicated on the surveys that they were somewhat confused about what the Capstone is and what their options are. Therefore, we will include more information on this both at the student orientation, and on the Psychology Dept. web site.

7. Future assessment. For next year’s assessment, we plan to give the Capstone survey to a greater number of students. We will also give a similar survey to students who have not yet done their Capstone project so that later, after they’ve done it, the results can be compared.

We would also like to determine whether the changes made in Psyc. 200 have had a positive effect. To do this, we will compare the distribution of grades, and the percentage of students dropping the course, before and after the changes were made.

Finally, we would like to examine Goal B (understanding of scientific method) in more detail. We plan to develop a test that can be administered to students at the beginning and end of Psych 202, Psyc 302, and the various Lab courses. The results of this test will be used to track students’ progress over the course of their time with us (Psyc 202 is usually taken in the Sophomore or Junior year, followed by Psyc 302, followed by Lab in the senior year). The results of the tests given at the beginning of a course can also be used by the professors of those courses to let students know if they appear to be lacking in pre-requisites.