Hub innovations: Outcomes measurements

Hub innovations: Outcomes measurements 

Evaluation is a key element of any educational program (1). It is essential to understand how effective an educational activity has been in closing the practice gaps identified in the needs assessment. To evaluate changes in knowledge and competence, we design pre- and post-activity questions to ascertain whether our specific and measurable learning objectives have been met. The pre-activity test questions are identical to the post-activity test questions, so by using the pre-activity test responses as a baseline, we can measure improvements resulting from our education. For our hubs, these pre- and post-activity questions take the format of a “Quick Quiz” intended as a fun challenge for readers, providing them with the opportunity to test their knowledge, improve on their pre-activity quiz score, and compare themselves to the average score of their colleagues. As the first step in the role out of the Quick Quiz, this functionality was introduced to the Lymphoma Hub earlier this year. 

Since the end of February 2022, we have published seven articles on the Lymphoma Hub that each include a precise learning objective and two pre- and post-activity questions. Initial data are encouraging, indicating high engagement and completion rates compared with conventional online content (2,3) and importantly, improvements in learners’ knowledge following review of our content.  

Key statistics 

• The engagement rate (number of page viewers that go on to play the quiz) has reached up to 24%. 
• For the seven articles published to date, the overall completion rate of the quizzes (percentage of readers who complete the quiz after initial engagement) is 73% (n = 62).  
• Importantly, of the completed quizzes, knowledge increased following engagement in 60% of the completed quizzes. No overall decrease in knowledge was recorded.  
• For quizzes where a knowledge increase was recorded, the average increase in knowledge was 49%. 

Excitingly, these data demonstrate the effectiveness of our education and the positive impact we are having on readers’ knowledge about specific elements of lymphoma management. The data will feed into our needs assessment process to ensure that we are continually optimizing our education. Analysis of the pre-activity test answers will help us to understand where educational needs are high, and analysis of the pre- and post-activity data will highlight whether our education is sufficiently addressing the needs identified in our needs assessment or whether further education is required in specific areas.  

Based on the success of the Quick Quiz on the Lymphoma Hub, we are excited to announce that quizzes are now available on all our hubs—try our recent quiz on the Multiple Myeloma Hub. Furthermore, we plan to introduce innovations to further increase engagement and provide benefits for individual users. At SES, our ultimate goal is to positively impact patient health; enhancing our capacity to measure and understand the outcomes of our education is a key step towards this goal.  

1. Shannon S. Education and practice. Needs assessment for CME. Lancet. 2003;361:974. 
2. CXL. What is a “good” click-through rate? Click-through rate benchmarks. https://cxl.com/guides/click-through-rate/benchmarks/. Accessed May 24, 2022.  
3.  Grow my Ads. The GoogleAds benchmarks you need to know in 2022. https://growmyads.com/google-ads-benchmarks. Accessed May 24, 2022.