Assessment and evaluation, item analysis included

A nurse educator develops a Likert-scale survey of learner attitudes toward interprofessional teamwork. Which reliability statistic is appropriate, and what does a value of 0.75 indicate?

1. Kuder-Richardson 20; acceptable consistency
   KR-20 applies only to items scored right or wrong, so it is the wrong statistic for Likert ratings whatever the value.
2. Cronbach's alpha; too low to use
   Right statistic, wrong reading: 0.75 clears the usual .70 threshold. Demanding .90 is a standard for high-stakes tests, not for an attitude survey.
3. Cronbach's alpha; acceptable consistency
   Cronbach's alpha estimates internal consistency for items with more than two score points, such as Likert ratings, and about .70 or higher is conventionally acceptable.

During skills practice, a nurse educator tells a learner, "Good job." Which revision would make this feedback most effective?

1. "Good job; I'll send your full evaluation at the end of term."
   Promising feedback later keeps the praise vague and delays it until it is too late to use in this practice session.
2. "Your hand stayed above the sterile field the whole time."
   Effective feedback names the specific observed behavior right away, so the learner knows exactly what to repeat.
3. "Great work today—you're one of the strongest in the group."
   Comparison with peers is norm-referenced and personal. It says nothing about which behavior was done well.

An item on a nursing exam has a difficulty index of 0.50 and a discrimination index of 0.45. How should the nurse educator classify the item?

1. An item too hard for this group to keep
   Difficulty p is the proportion who answered correctly, so 0.50 is moderate, not hard. Calling it too hard confuses difficulty with failure.
2. A well-performing item to retain in the bank
   Moderate difficulty (p = 0.50) combined with strong discrimination (0.45) is what a good item looks like, so it stays in the bank.
3. A poorly discriminating item to revise
   A discrimination index of 0.45 is very good (Ebel: 0.40 or higher). Reading it as poor reverses the scale.

A course coordinator proposes grading the clinical nursing exams on a curve. Which concern should the nurse educator raise?

1. Curving can fail learners who meet the standard
   Clinical competence is criterion-referenced. A curve ranks learners against each other, so a learner who meets every standard can still fail.
2. Curving may weaken healthy competition among learners
   A curve increases competition, because one learner's gain is another's loss. This concern also misses the real problem with norm-referenced grading.
3. Curving would slow grading for the course faculty
   Workload is not the issue. The concern is validity: grades on a curve no longer say whether a learner is competent.

On a unit exam, 85% of learners answer an item on diabetic ketoacidosis incorrectly. Which is the nurse educator's first action?

1. Remove the item and rescore the exams
   Deleting before analysis may throw out a sound item that exposed a real learning gap.
2. Schedule another lecture on DKA content
   Reteaching assumes the content was not learned, but a flawed item can produce the same result. Analyze first.
3. Review the item and its statistics for flaws
   When most learners miss an item, first check the item itself: miskey, ambiguity, fit with the objectives and the item statistics. Only then decide whether the problem is teaching or the test.

A nursing program must set a defensible cut score on an exam used to certify graduation competence. Which principle should guide the standard-setting process?

1. Match it to prior cohorts' historical pass rates
   Anchoring to past pass rates bakes in earlier cohorts' performance instead of a standard of competence.
2. Set it so a fixed percentage passes each year
   A fixed pass percentage is norm-referenced: the cut moves with each cohort, so it no longer certifies competence.
3. Base it on experts' borderline-candidate ratings
   A defensible cut score for a competence decision is criterion-referenced: experts picture the borderline, minimally competent candidate and judge each item against that standard, as in the modified Angoff method.

An item on hand hygiene, a core safety competency, has p = 0.95 and a discrimination index near zero. Which action is most appropriate?

1. Retain it as a mastery item on must-know content
   Mastery items on must-know content can have p near 1.0 and little discrimination. They stay because they sample critical content, not because they spread scores.
2. Rewrite it to be harder so that it discriminates better
   Adding difficulty to safety content tests trickery rather than the competency every learner must have.
3. Delete it because it does not discriminate
   Low discrimination is expected when nearly everyone answers correctly. Deleting the item drops must-have content from the test.

A nurse educator must combine written exam and observed skills station results into one course decision. Which approach best preserves validity?

1. Average the two scores into one course grade
   Averaging is compensatory: strong exam scores can offset unsafe skills, which undermines a competence decision.
2. Require a minimum pass on each component
   A conjunctive rule, a pass on each component, ensures the decision reflects both knowledge and observed skill and blocks compensation.
3. Use whichever of the two scores is higher
   Taking the better score lets one strength hide a failed component, so the decision no longer reflects both.

After a clinical skills assessment, which feedback approach most strongly promotes learners' self-regulated learning?

1. Peer feedback in place of faculty feedback
   Peer feedback can add to faculty feedback but not replace it, and on its own it does not build a plan for improvement.
2. Specific feedback plus a learner action plan
   Self-regulation grows when learners turn specific feedback into their own goals and strategies, which is what an action plan does.
3. Detailed feedback given along with the final grade
   Feedback attached to a final grade arrives when nothing can be changed, and learners tend to look at the grade and skip the comments.

On a unit exam, three items on fluid balance had low p-values and strong positive point-biserials, and learners commented that the content was rushed. Which is the nurse educator's best action?

1. Rewording the three items so they are easier
   Making sound items easier hides the learning gap instead of closing it.
2. Deleting the three items and rescoring the whole exam
   Deletion is for flawed items. These items discriminate well, so removing them discards valid evidence.
3. Reteaching the topic and reassessing learning
   Strong positive point-biserials show the items work: the stronger learners got them right. Low p plus 'rushed' points to a learning gap, so reteach and reassess.

A clinical instructor is writing anecdotal notes about a learner whose professional behavior is a concern. Which entry is most appropriate?

1. Arrived late on 3/4 and 3/6, showing poor time management
   The dates are specific, but 'showing poor time management' adds an inference. Anecdotal notes record what was observed and leave the judgment out.
2. Arrived at 0720 for 0700 shifts on 3/4 and 3/6 without calling
   A good anecdotal note is objective, specific and dated: times, dates and what the learner did, with no judgment added.
3. Showed a dismissive attitude toward staff nurses during morning report
   'Dismissive attitude' is an interpretation, not an observed behavior. Anecdotal notes record what was seen and heard.

Which scenario best illustrates formative assessment in a nursing course?

1. A graded midterm exam that counts for 25% of the grade
   A graded midterm that counts toward the course grade is a judgment of attainment, which makes it summative even mid-course.
2. A skills check-off that certifies clinical readiness
   Certifying readiness is a pass/fail decision on attainment, so the check-off is summative.
3. A weekly ungraded quiz used to adjust the teaching
   Formative assessment happens during learning and feeds back into it. An ungraded quiz used to adjust teaching does exactly that.

A faculty member argues that a final exam is valid because its KR-20 is 0.92. Which evidence would best support the validity of the exam's score interpretations?

1. Expert review that maps the items to the blueprint
   Content evidence is the core validity evidence for a teacher-made test: experts confirm the items sample the blueprint's content and cognitive levels.
2. Learners' comments that the exam looked fair
   Face validity, how a test looks to test-takers, is not accepted evidence for interpreting scores.
3. A second administration with a similar KR-20
   A matching KR-20 shows the scores are consistent again. Reliability is necessary for validity but is not evidence of it.

On a unit exam taken by 50 learners, 45 answer one item correctly. Using a target difficulty range of 0.30 to 0.80, how should the item be interpreted?

1. p = 0.90; easier than the target range
   p is the proportion answering correctly: 45/50 = 0.90. A higher p means an easier item, so 0.90 is above the 0.80 ceiling.
2. p = 0.10; harder than the target range
   0.10 is the proportion who got it wrong (5/50). The difficulty index counts correct answers, so this reverses it.
3. p = 0.45; well within the target range
   Reading the raw count of 45 as the proportion gives 0.45. The difficulty index is 45/50 = 0.90.

A nurse educator must evaluate both sterile technique and clinical reasoning during a urinary catheter insertion. Which strategy best captures both in one performance?

1. A checklist observation with think-aloud questions
   Observing the skill against a checklist while the learner explains each decision aloud captures technique and reasoning in the same performance.
2. A written exam and a separate return demonstration
   Two separate measures cover both domains but never in one performance, so the link between reasoning and action stays untested.
3. A self-reported insertion log reviewed by faculty
   A log records what the learner says happened. It is not direct observation of either technique or reasoning.