Beyond session evaluations: Onsite digital learning assessments that go the extra mile


The acquisition of Continuing Medical Education credits and Maintenance of Certification points is a key element of how physicians demonstrate dedication and commitment to their ongoing ability to provide high-quality healthcare. Increasingly, associations and societies are under pressure to provide educational content at their meetings that includes the ability for physicians to get credit/points for attending. It’s critically important that our world’s citizens have affordable access to modern, innovative healthcare, and keeping busy physicians engaged in meaningful life-long learning should matter to all of us (cue rainbow machine, sparkly butterflies, and unicorns).

But as an accreditation committee or meeting professional, have you ever been in the hot seat for actually making this happen?

Here’s a sample process for consideration (feel free to skip ahead if you’re short on time):

  1. You must engage your accreditation committee in reviewing your meeting’s content and selecting appropriate sessions/activities for which you want to give credit.
  2. You have to get someone (or a group of someones) to write credible, rigorous, accreditable assessments for those sessions.
  3. You then have to publish these assessments, distribute them onsite, and collect them onsite.
  4. Once collected, you have to proceed with reviewing and scoring the assessments. As part of this, you must have some kind of system by which you are keeping track of which physicians took tests for which sessions on which dates at which time.
  5. You distribute the assessments to various committee members who are responsible for scoring them, and then need to collect the finished scores when the process is complete.
  6. Only then, you can cross check the finished scores with your original list of applicants and start letting them know whether they passed or failed.
  7. Of course, you also have to track down and nag any reviewer/scorers who may be behind in their reviews.
  8. You also need to answer questions and complaints from your learners along the way — including providing further details for those who may not have passed their assessments.
  9. Finally, you will need to create certificates and documentation for those who passed, and then…
  10. Create and send reports to the official accrediting agencies.

Are you still with me? OMG.

There’s a reason why many societies don’t offer CMEs and MOCs for attending their conferences, and it ain’t that they don’t believe in the value of continuing education. It comes down to a handful of simple factors: time, money, staffing, and onsite resources.

But what if that process was digitized and automated from the point where the session assessments were designed? What if conference attendees could simply filter and sort conference schedules from the palms of their hands, selecting and scheduling themselves to attend sessions for which they could take assessments and get credits? And what if they could actually take those assessments with a digital device of their choice (smartphone, tablet, laptop) and get instant feedback on their performance? And if they didn’t pass, what if they could in that very next instant retake the assessment? What if after the conference, all those who did pass one or more assessments received a customized report and documentation of their performance, accompanied by an official certification?

And what if all of this (from Step 3 to Step 9) happened with very little, if any, staff impact —and it took minutes not weeks. Would that be valuable to you and your society?

Enter a groundbreaking collaboration between the Heart Rhythm Society and CTI Meeting Technology that resulted in HRS’ ability to increase its 2016 CME/MOC awards by more than 800 over the previous year. Working with CTI, HRS was able to award a total of 119 MOC points to 239 attendee physicians. More than 933 tests over 16 MOC sessions were administered in total, with an average of 2.6 points and 3.9 tests per physician.

The medical meetings industry is standing up and taking note of innovations in CME and MOC administration.

Quoted in Medical Meetings Magazine’s September 2016 issue, CTI CEO Mark W. Coe says, “{Societies} have to think about session development a little bit differently.”

“HRS is the first society we worked with to take the workflow all the way through,” Coe says, though “almost everyone we talk with is interested in and thinking about doing this. Being able to integrate MOC into the existing workflow means you don’t have to hire yet another vendor to come on site.”

Also quoted in MMM:

“Physicians need MOC points for the certification and recertification. It’s neat to see them come to us, to participate and be active in the process,” says Tracy Blithe, CMP, HRS Manager, CME Programs and Meetings Operations.

Adds Chloé Thomas, HRS Director, Scientific Sessions Programs and Education, “So far, everyone we have spoken with has commended us for the effort of streamlining MOC points and CME credit.”

Muzik & Muzik’s “In a Nutshell” perspective: This is only the tip of the iceberg. While certifications and continuing education in medical ed will always be hot topics when it comes to assessing learning, there are endless possibilities for how this kind of technology collaboration can impact the efficiency with which professional organizations can connect their event content with measureable learning outcomes.


Learn more about HRS’ success at

Learn more about CTI Meeting Technology, the company behind HRS’ technology solution at:



Pragmatic Marketing: For the Love of Learning

I’m learning today.

Okay, I learn EVERYDAY. But what I’m doing today is especially exciting: I’m learning more about the Pragmatic Marketing Framework.

Until last week when it was mentioned by a friend at CTI Meeting Technologies, I had never heard of it. But as of 15 minutes ago, I am on fire.

Essentially, the PMF says that we should stop wasting time persuading people to buy what we offer. Instead, we should find out what they want and then build that. To do this, we have to have a whole lot of discipline around us. We have to stop doing the wrong things and start doing the right things. The PMF helps us identify those things and get moving forward with the process of making it happen.

What I am excited about and find so unique is the packaging of this concept. The framework itself is crystal clear. The interactive graphic on their website is glorious in its simplicity — and is the Google maps to your marketing journey.  Click through to it, as definitions for each of the blue squares can be seen with a roll-over.



When I was in undergraduate school at the University of Oregon’s School of Journalism, in their then “creative advertising” track, we learned that it’s always far easier to sell someone something they want rather than convincing them to buy something they don’t want or need. Beauty product marketers have known how to do this for years. Women want to look younger…so companies design and sell products that say they’ll meet that need. The global cosmetics industry is estimated to reach 675 billion USD by 2020, according to Research and Market’s Business Wire report from July 2015.

Steve Jobs had his own perspective on this, with his quote about Apple’s success coming from selling people things that they never even imagined. While knowing your customers so well that your products sell themselves is incredible , most of us don’t have 30 years to build a brand from a garage business to a stratospheric success. Rather…we need sales now!

There are 37 discrete strategies to the PMF, which seems like kind of a lot. However, none of them are complete mysteries, as most of us do at least some of them some of the time. They have origins in strategic planning, SWOT analyses, marketing planning, sales, and most of what we learn in business school.

“Essentially, the PMF says that we should stop wasting time persuading people to buy what we offer. Instead, we should find out what they want and then build that.” CLICK TO TWEET

Tools are no good if you don’t know how to use them, but good tools in the right hands are transformative.

As a consultant, kicking off a new relationship knowing that a staff has been coached and trained in a common framework of any kind is exciting. Once you learn the language of the framework, you can step in and apply the years of experience they are paying you for with a much higher prospect for lasting success.

In my case, I’ve been fighting uphill often to convince clients of strategies for which their entire internal structure is underprepared. Creating common ground so that we’re all pulling in the same direction takes a hellishly long time. Yet success without that is hard won and expensive.

I am over the moon thinking about how the structure and principles set forth in the Pragmatic Marketing Framework can be leveraged for even greater client success than I imagined.

Hat’s off to learning new things!


And now for something different…Introducing YOOL

Sometimes it’s fun to mix it up a little.

After a spring and summer chock full of eLearning deep dives, M&M took a little break and visited the world of software development. Beginning in the fall, I got to drive product and brand development for a potentially disruptive and transformative new arts administration tool — called YOOL. Last week, our website finally went live and we’re getting busy with preparations to launch our first sales campaign! Woot woot!

A quasi-acronym for “Your Orchestra…Online,” YOOL was designed by orchestra administrators FOR orchestra administrators, as what essentially boils down to as a highly customised productivity suite of cloud-based tools. Perhaps it’s a sort of über Microsoft Office combined with a rockin’ library of digital assets, messaging, and payroll capacities.

At any rate, YOOL is poised to really make a difference for those in arts administration roles who are seeking ways to do more with less. No one has enough time, staff, or brainshare anymore, and I personally believe that the arts are more important than ever to our society. YOOL is a way for directors and creatives who lead companies, symphonies, orchestras, and ensembles to efficiently manage the day-to-day stuff that goes into any one single performance. It’s a chokingly huge amount of detail to manage, and YOOL helps its users share, update, archive, and plan all within a single interface

“Working with an orchestra using YOOL is a dream come true.” CLICK TO TWEET

Most of us just simply go to the symphony, opera, or philharmonic and have no idea that each one of those musicians must be auditioned, rehearsed, contracted,  scheduled, paid, and communicated with hundreds of times before they ever appear in concert. YOOL’s interface manages all of that, allows artists to create individual profiles where they can store copies of their travel documents, contracts, scores, publicity photos, schedules, and instrumentation.  For a busy musician who wants to stay as far away from a desk of paperwork as possible, working with an orchestra using YOOL is a dream come true.

This kind of technology exists in many fields — think CRMs in the business world or even LMS’ in the education space. Yet, the arts have been slower perhaps to develop and embrace the very tools that can free them up to create MORE and fuss with paper LESS.

I write about YOOL here not just to crow on a very cool product…rather I write about it as a reminder that taking on new stuff can be really exciting. And taking a break from working on the usual to dip into something completely different feeds me and my work in ways that I sometimes forget are necessary to stay fresh and relevant. Pioneering isn’t easy, and it can be really exhausting. But it also brings with it a feeling of accomplishment and delight that I know perks up my step and brightens my eyes. It also opens me to learning opportunities and a wider outlook on the ever-changing world around me.

Do you have a YOOL in your life? What is it and what difference does it make to you as a learner and producer?


Transforming Credentialing with Digital Badges: CESSE 2015

For those who attended “Transforming Credentialing with Digital Badges” last week at CESSE 2015 in Norfolk, here is a pdf download for your perusing pleasure! Please be sure to give intellectual credit when using beyond your own personal reference — a reminder that Dr. Tracy Petrillo from EDUCAUSE, and the research team of Dr. Daniel Hickey and Dr. James Willis from Indiana University were contributors to what you see here.


As always, please leave your thoughts and contributions in the comments area below, and don’t hesitate to reach out directly with any questions!


Engage Learners by Giving Them What They Want

Thanks to everyone from the #CESSE2015 attendees in the Education & Training and Publications tracks for attending yesterday’s session on needs assessments. What a great conversation and what a delightful audience! You made it really fun to lead our conversation 🙂

For those who’d like to follow-up or who didn’t have a chance to attend, here are the slides from that presentation — if you want to use these for any purpose other than your own personal reference, please be sure to give intellectual credit to Muzik & Muzik (and/or other references), and please drop me a line or comment to let me know how you’re using them! Note: This is an automatic PDF file download of about 2 MB.


If you’d like to discuss how M&M can help you with structuring, administering, or analyzing a needs assessment within your organization, company, or department, please let us know!

Getting our conference on: CESSE Annual Meeting in Norfolk

In just one short week, M&M will be taking part in the Council of Engineering and Scientific Society Executives (CESSE) annual meeting in Norfolk, VA, July 14–16, 2015. It’s been a pleasure to serve as members of the Education and Training track this year, along with some really wonderful and dynamic colleagues!

Last July (2014) was M&M’s first experience working with CESSE, when we participated as conference sponsors. In this capacity, we had the opportunity to take part in the full complement of sessions, networking events, and meal functions — as well as the sponsor marketplace. It was a perfect first dip into the CESSE pool and we were almost instantly recruited to participate as content contributors for 2015. A huge win for us and we were really excited.

As a member of the 2015 CESSE Education and Training program committee, Donella is leading/chairing two sessions on M&M’s behalf:

Engage Learners by Delivering What they Want: Focus on Needs Assessments, Wednesday, July 15, 11:30 am–12:45 pm (joint session with Publications track)

Transforming Credentialing with Digital Badges, Thursday, July 16, 11:30 am–12:45 pm

Mathias Posch, President of ICS (International Conference Services) Events will join in on the session on needs assessments (“Engage Learners…”), sharing his perspectives and experience on working with a number of global associations to design and administer needs assessments that lead to the development and delivery of functional and effective programs.

Dr. Tracy Petrillo — the Chief Learning Officer for higher-ed powerhouse organization EDUCAUSE — will lend her voice and expertise to our session on digital badges. Tracy has been part of EDUCAUSE’s efforts to establish badge programs in support of both member engagement and continuing education and was a featured panelist at 2015’s SXSW-EDU conference in Austin.

If you’re attending CESSE in Norfolk, please check back to this blog for additional pre-session postings, resources, and follow-ups! See you there!

Karen Hyder on Shifting Delivery for Virtual Learning

via Technology & Training: Karen Hyder on Moving to Virtual Classrooms.

For those who attended our recent talks at PCMA’s Convening Leaders and IAPCO’s Wolfsberg Seminar, you might remember discussion about looking to the “learning sciences” for the next big trends in hybrid meetings/events and content management/delivery.

Ms. Hyder’s blog post on the move to virtual classrooms presents an appropriate and welcome emphasis on the shift from face-to-face to “virtual” learning spaces.