Time off for professional developement

My blogs on ID standards will continue next week.  I’m taking the week off for professional development. I submitted my application for an instructional designer certification in synchronous learning (ID:SEL) this past weekend. And, I am working on an application for Certified Assessment and Credentialing Professional.  Then I need to renew my CPT.

And, I am attending Practical Analytics and Psychometrics with Drs. Judith Hale and Jobie Skaggs, which has some really great stuff for both certification geeks and instructional designers. An entry point discusses different types of measures. Do you know the difference between analytics, psychometrics, evaluations, and pragmatics?  

Analytics

Measures the impact on business or, in the case of a certification, on the public promise. 

For example, many certification imply that the certificant will be able to earn a better living because they are certified; that is their public promise, whether it is stated or simply implied.  The analytics this organization may need will be related to salary, position titles, promotions, ability to get consulting contracts, etc.  This data will then need to be compared against the general field to determine whether the public promise is being met.

For learning organizations and instructional designers, the important analytics are business measures that learning interventions propose to impact.  Do people who attend your training decrease turn-around time, decrease waste, increase sales, improve quality, etc.?

Analytics are measures of the environment – frequently business measures specific to the industry, particular business organization, or certification’s public promise.

Psychometrics

Measures of the assessment instruments.  Answers questions about whether the test measures what it purports to measure, whether it discriminates between the qualified and less than qualified individual, for example.  The test types and statistical manipulations often get complex and esoteric – item analysis P scores, pass score (standard) setting. (The workshop goes into more detail here.) However, the trick here is to have a psychometrician or statistician who can help you define the test type you need for your situation, run the numbers, and then help you understand them.     

Evaluation

The questions and judgments you (and your leaders) make based on the numbers generated through analytics or psychometrics.    

Evaluations answer value questions.  Is this where we wanted to be? Is it right or good?  Could we do better?  Is something wrong?  Are we moving in the right direction?

Pragmatics

At many points throughout the journey, pragmatics come in to play.  Here, “real world” factors influence decisions.  These may be time and money factors, availability of resources, political, or simply personal preference. Pragmatics influence decisions on what to measure, whether to tackle a more complex measurement in order to have strong numbers for future decisions, whether to accept resulting evaluations, whom to include in building the assessment tools, and much, much more. 

Need more? 

Want to join the next session, go to www.tifpi.org and search the available workshops and visit the Certification tab, Certified Assessment & Credential Professional.