IDs Use Standards: Assesses Performance

Standards are different from theories or models; they are internal lens that competent IDs use when evaluating their effectiveness and quality of the learning solution they develop.  Each ID applies many lens to their work as he or she moves through the cycle that is learning solution development.

Standards are the measures that IDs use when determining whether they will sign-off on a learning solution, or not – whether their name goes on the final product.  What standards do you use?

The competent instructional designer/developer (ID) assesses performance during learning:

For many, the word “assessment” equals “test”, preferably a multiple choice test.  However, there are many kinds of assessments –observation, track learning outcomes (mastery),
measurement, checklists, attestations, record assessment (e.g., polls, surveys, self-
assessments, tests, interactive activities in elearning modules, checklists, observation 
worksheet, etc.) and many different blends of those techniques.

Assessment is challenging under any condition and becomes more so within the boundaries of the learning environment.  Assessment is the ID’s opportunity to determine whether learners developed new skills and knowledges.   In creating assessments, the ID must consider 

the limitations and unique opportunities of the learning solution.  For example, self- 

contained asynchronous elearning has different assessment opportunities than does a 

coaching situation or a simulation.  In one case, there is a computer to track and measure; 

in the other, there is a human.  Both bring unique characteristics to the assessment.  Either way a rubric needs to be developed to ensure fair evaluation of all learners. 

Once the assessment tools and related rubrics have been developed, they must be tested themselves.  That is, a part of ID’s job is to test the assessment to ensure that the assessment really does assess the learning outcomes and that the scoring rubrics are fair. 

Assessment is one of the more complex portions of the ID’s work, and, often, one of the least valued. 

Case Study:  

Once upon a time, a relatively experienced ID got the assignment-of-her-life
 – to build a goal-based scenario for 24 non-traditional software programmers who would be doing code maintenance (a different process than creating new code).  The participants would be non-traditional programmers – individuals without college degrees in computer science who had received an intensive 
8-week programming language immersion.  The goal-based scenario would provide them with a mock environment of the work they would be doing and would assess their strengths in order to place them appropriately within the organization.

In a way, a goal-based scenario (also called a war-game) is a one big assessment supported by a resource-rich environment that includes text, video, 
coaching, process support, and more.  In this case, the participant 
programmers received six or seven mini-scenarios called a service ticket point (STP). The STPs were handed out over five days, so that participants never had more than 3 STPs at once, and never less than 1 STP.   Each STP reflected a typical code 
maintenance issue.  The task was for each participant in this onsite classroom-based 
war-room to identify each code issue, resolve it and document the resolution. 

Assessment included a checklist that coaches used to review the work with each individual.  The rubrics for this coached assessment could be standard across all service ticket points, 

but each mini-scenario had different skills and knowledges to demonstrate through 

resolution of the problem.   

The real challenge for this assessment had to do with the technology itself.  In order for two dozen students to simultaneously access a software system to identify and resolve a problem, each one of them had to have a separate instance of the software.  In order to have the software, available with the same problems in each instance, a programmer had to capture one version of the software and “back out” all the revisions that had bug fixes for the problems identified.  Then twenty-four copies of that older broken software environment had to be installed so that they did not conflict with current software environments.  Once installed, each had to be tested to be sure that the code was broken in the key spots and that that instance did not conflict with other instances.  Once those broken software environments were available, participants could apply their newly developed programming language skills to solving the problem.  Event coaches (expert programmers) could see how each participant had resolved the problem and could provide feedback on the way that the code was written. 

Defining the assessment environment, setting it up, and testing it was key.  However, the ID was not a programmer herself.  The ID’s role was to get the experts in these environments to do the detailed work of preparing the broken software instance, replicating, and testing it. 

Individual learners acquired a portfolio of scenarios that they had solved.  Some were able to solve all seven scenarios in five days, while others only completed five scenarios (the minimum requirement).  By allowing learners to work at their own speed, the coaches learned more about each participant’s approach to problems.  These insights helped the coaches recommend placement on specific software teams. 

This was a very complex learning and assessment solution with its complexity starting at the beginning – the need for participants to build skills in programming and problem-solving and then demonstrate that they could do the work. The complexity continued through the development of mock service tickets, the coaching evaluation rubrics, preparation of the system, preparation of the coaches, and validation that each individual completed at least the five minimum service tickets.   

In addition, the work was real-world work that could be assessed in a standardized way.  That assessment results were used to assist with placement of the newly minted coders was a bonus.  

Definition of a Standard

Consider the definition and performances listed for The Institute for Performance Improvement (TIfPI’s) Assesses Performance (during learning).

Definition:  evaluate what the learner does within the learning environment using a specific set of criteria as the measure or standard for the learner’s progress.

Performances that demonstrate this standard for a Solution Domain Badge:

·         Creates metrics or rubrics that guide the assessment of performance within the learning environment

·       Creates effective assessment tools⁽¹⁾ to support the assessment process.
·       Creates instructions for using the performance tools.
·       Pilot tests tools to assure that the tool measured the appropriate performance.
·       Modifies tools based on feedback from pilot testing.
·       Ensures that resulting data drives feedback to the learner, to the instructor, to the

     sponsoring organization, or to the instructional design process for future modification. 

(1)      Assessment tools may include any technique to observe, track, measure, or record assessment (e.g., polls, surveys, self-assessments, tests, interactive activities in elearning modules, checklists, observation worksheet, etc.)

Note that any one solution may not require the use of all 6 performances listed.  Individuals applying for learning solution badges will be asked to describe how they demonstrated at 

least 3:6 performances, two of which must be:

o   Creates metrics or rubrics that guide the assessment of performance within the learning environment.

o   Creates effective assessment tool(s) to support the assessment process.  

Can you see yourself doing these performances?  Can you see yourself doing at least 3 of these performances with every learning solution?  Can you see other IDs doing these performances, perhaps differently, but still doing them?  If so, you need to consider becoming 

certified in learning solution development.  Attend the next Overview of ID Badges free 

webinar.

Want a list of all 9 ID standards?  

Would you like to know about the study -- a practice analysis -- that TIfPI Practice Leaders did to generate and validate nine standards, including Assesses performance?

www.tifpi.org 

ID Standards: Aligns Solution

Standards are different from theories or models.  Standards speak to the ways that competent professionals judge their own work and that of their peers.  Yes, the interwoven nature of standards, theories, and models gets tangled and convoluted. We are discussing here the internal standards that competent IDs use. That those standards emerged from theory and development models is true, but they are different from both theories and models.  For IDs, these standards are also international in nature, being used by learning experts around the world.  Think of these standards as lens describing the effectiveness and quality of the learning solution.  Each ID applies many lens to their work as he or she moves through the cycle that is learning solution development.

Standards are the measures that IDs use when determining whether they will sign-off on a learning solution, or not – whether their name goes on the final product.

The competent instructional designer/developer (ID) aligns the solution:

Alignment has become a buzzword. Amazingly, it looks different from different angles.  However,
different perspectives really does mean that alignment is a key function of learning solution
development.  It’s is a real function of the work.

In fact, this the standard was rated as very highly important in a survey of learning practitioners,
where it received an average of 3.9 out of 4.0 points. 

Obviously, one of the challenges in creating learning solutions is aligning them to the needs of the organization and the learner.  This external alignment ensures that the learning solution is really needed and will be used.  However, alignment does not end there.

IDs building learning solutions check, double-check, and triple-check that the parts of each learning
solution work together.  They check that learning objectives actually guide the learning – that they
are do-able and actionable. They ensure that those objectives really are the work outcomes needed on the job.  They check that learning in one solution connects appropriately learning in another solution.  If all the elements are not aligned, they modify the solution to create better alignment. 

Case Study:

An executive working with an external ID took exception to the word “align”
in a learning outcome.  She said that the word made her think of a pilot lining
airplane up for landing or bringing a ship into dock – that it was a purely
physical act, like hammering nails, and not at all intellectual. She changed the
outcome to “understands”. 

 

Some executive decisions are the other kind of alignment – aligning the
learning with the needs of the organization.  In this case, the alignment
adjustment needed to be using a verb from Bloom’s Taxonomy because higher education is stuck with that paradigm and judged on their use of a limited list of verbs.    No matter how much this expert ID distrusts objectives using the verb “understand”, the objective needed to fit the organization’s accreditation
requirements even more than her personal standard for objectives, and this organization was
comfortable with the use of understand as a knowledge-testing function rather than a performance-
assessment function.

From the beginning to the end of a learning solution development project, IDs drive for better
alignment whether that is alignment of activities and assessments to objectives (internal alignment
within the course) or alignment of the learning solution to the organization’s needs (external to the
course) or alignment between courses (curriculum alignment).  One and feel that that the prize has
been won, when the learner experiences internal alignment that blows them away.  

Definition of a Standard

Consider the definition and performances listed for The Institute for Performance Improvement (TIfPI’s) Aligns Solution

Definition:  To create or change relationships among parts of the solution (internal to the solution) or between the solution and its parent organization or sponsors (external to the solution).

Performances that demonstrate this domain for a Solution Development Badge:

·         Maps the instructional elements to defined project and audience requirements.

·         Sequences learning elements and content appropriately for defined learners.

·         Modifies planned instructional elements in order to make those elements more effective.

·         Selects appropriate content for the solution.

·         Maps content to appropriate instructional elements. 

Note that any one solution may not require the use of all 5 performances listed.  Individuals applying for learning solution badges will be asked to describe how they demonstrated at least 3:9 
performances, one of which must be:

·         Maps the instructional elements to defined project and audience requirements.   

Can you see yourself doing these performances?  Can you see yourself doing at least 3 of these
performances with every learning solution?  Can you see other IDs doing these performances,
perhaps differently, but still doing them?  If so, you need to consider applying for a learning solutions development credential.  Go to https://tifpi.wildapricot.org/IDBadges.

Want a list of all 9 ID standards?  Go to http://tinyurl.com/nqjwm2g.

Would you like to know about the study -- a practice analysis -- that TIfPI Practice Leaders did to generate and validate nine standards, including Aligns Solution?  Go to http://tinyurl.com/pd69xw5.

ID Standards: Addresses Sustainability

Standards are different from theories or models.  Standards about the ways competent professionals that judge their own work and that of their peers. 

For instructional designers (IDs), theories are filters that create mental models and structures for what learners need to have in place in order to learn.  As such, theories are not standards.  However, they do create a common language around the application of that theory.  Over time, theory-in-practice creates a mental model about how that theory should play out when applied.  IDs have learned how to mix-and-match theories to take the best of multiple theories in order to create better and better learning solutions.  We (for I am an ID, as well) have created an internal set of standards that cross multiple theories.  The theories are not the standard, our mental model of combined theories is.

Similarly, product and instructional solution development models such as ISD, ADDIE, SAM, Agile, Lean, or Six-Sigma proscribe techniques for moving the learning solution idea from idea to delivered learning.  They create a common language around the process of building learning solutions.  

Experienced IDs use the stages of development and tools of those stages as markers to guide them in determining whether the emerging learning solution meets their own personal standards.  

Yes, the interwoven nature of standards, theories, and models gets tangled and convoluted.

We are discussing here the internal standards that competent IDs use. That those standards emerged from theory and development models is true, but they are different from both theories and models.  Think of them as lens describing the effectiveness and quality of the learning solution.  Each ID applies many lens to their work as he or she moves through the cycle that is learning solution development.

Standards are the measures that IDs use when determining whether they will sign-off on a learning solution, or not – whether their name goes on the final product.

The competent instructional designer/developer (ID) addresses sustainability (of the learning solution).

At first, "sustainability" seems to be a lesser standard than, say, Aligns Solution, Engages Learning, or Assesses Performance. As the ID Badge Team at TIfPI worked on defining ID standards and performance, this standard seemed to not fit, even though surveyed practitioners indicated that they did do this work frequently and that it was important.  Over and over again, this one was tough to define and describe.  Then, suddenly, it came into focus; it made perfect sense.  None of wants to engage in the development of a learning solution that will not be valued.  We value our work and our time, we want our products to be valued.  Therefore, we all address sustainability. 

Strangely, sustainability is something that instructional designers 

and developers address from the moment that they are engaged in planning a learning solution and continue to address throughout the development project right up to the end.  Yet, it is one that we do not talk about with other IDs.

Consider a typical engagement start.  A sponsor or project lead meets with the ID to introduce the project.  Right away, the ID starts asking questions about the purpose of the project and how it the learning solution will be deployed.  In the back of his or her mind, the ID is also asking themselves whether this is the right project for them, whether the resources are available, and whether the timeline is reasonable.  They are addressing sustainability of the learning solution project -- can I do this.  If it do-able, the ID moves on to other issues.  If not, he or she begins negotiating (time, content, tools, others involvement, etc.)

Even while trying to determine that the project is viable, competent IDs are also asking questions about the how the learning solution will be deployed.  They ask about the size of the audience, the connection between the course and the work, the preparation that learners will have before the course, and the follow-up ensuring transfer.  Here IDs validate both short-term sustainability of the course – that it can be deployed and delivered a few times.  They also check that there will be a long-term need for the learning solution -- that it solves a problem over time and that the first learners can demonstrate change so that future learners will want to come to the course. 

When the learning solution project has been negotiated and accepted, development beings.  While the negotiations and acceptance may be very short and very informal, they do exist for everyone. 

During development, competent IDs continue to work toward that end goal of a sustainable learning solution that meets the need and draws in learners over time.  This includes seeking appropriate content, identifying ways to improve the content, building more robust and less expensive courses, and creating a learning solution that is easy to maintain over time.

Case Study: the Case of the Unsustainable Learning Solution

Once upon a time, there was a young and idealistic ID called in to develop a course for a highly technical software that was still just an idea.  The sponsor was a respected scientist and corporate executive who wanted an instructor-led course that would teach programmers, data base administrators, systems architects, and content developers enough about the scientific field that they could build the software, database, interfaces, and manage complex content sources.  When asked about the details of the course’s audience, the ID discovered that each audience needed a slightly different class, as the software under development grew.  The course content stayed the same, but the tools used for activities would be different at different stages of the software development project.  The only instructor would be the scientist and the course would need to change between every delivery.  The size of the audience that would receive this highly customized courses?  Ten to fifteen people at each of five stages. 

Sustainability Question: Is it feasible to customize a course to this degree for an audience that is this small?  Probably not.  Yet this course was done. 

The point in addressing sustainability is to make the unwise or improbable solution workable and reasonable.

After some discussion, our ID discovered that the basic content and activities would be static.  Our ID discovered that course activities could be done through a variety of online tools (without the yet-to-be-designed software.)  The expert simply wanted learners to do exercises in the new software (in a DEV environment) as it developed, but continually emerging activities would have the same purpose as those done through the online tools.  Our ID also knew that new software systems often failed on the day that you really needed them to work, as well.  Therefore, using a DEV version of a software could make a course activity unviable. The key to sustainability was in the online tools. 

Solution: This course was developed only once using the online tools for individual activities. Then, it could be tweaked at every re-delivery of the course adding in equivalent activities using the DEV environment tools as those tools emerged. If DEV wasn't functioning on the day of the class, the original activities would continue to be available with the online tools.  Designed with only the smallest of tweaks between classes, the expert himself could modify the participant materials adding new versions for each activity while keeping the original version in the materials – adding exercise 2B, while keeping 2A.  During class the instructor could simply define which activity (2A or 2B) was being used that day based on whether DEV was functioning correctly or not.   Each round of delivery added one or more new activities (2B, 3B, 6B) depending on what had been developed and was functional at each stage.  Each course could be unique, while the overall work effort to maintain the course materials was low.

It worked.  It was a sustainable course over several years during times of high change.  Addressing sustainability moved this improbably course from 'can't do it' to 'done and working.'

Do ID's address sustainability?  Yes, all the time.  Every day.  Every project.  Every learning solution.

Definition of a Standard

Consider the definition and performances for Addresses Sustainability as listed for The Institute for Performance Improvement (ITIfPI's) ID Badges.

Definition:  Considers the best usage of resources (time, money, materials, staffing, technologies, etc.) now and in the future.

Performances that demonstrate this standard for a Solution Domain Badge (one or more of the following):

• Selects tools and methods that can be replicated at minimal costs and time.
• Builds in techniques that allow subject experts and instructors to modify the learning solution without requiring the solution to go through a complete revision cycle for each modification.
• Recommends tools and techniques that improve the learner’s learning environment and better match the learner’s needs.
• Recommends tools and techniques that improve the learning solution’s cost effectiveness.
• Leverages content, solution development processes, and solutions for reuse and for lowest cost of reproduction.
• Develops solutions that can be turned over to a different team that will support or teach it over time.
• Develops solutions that include planned future review cycles.
• Remediates expensive one-time solutions with follow-up that allows learners to access elements of that learning solution.
• Explains improvements to original learning design where such improvement created savings, improved learning, improved functionality, generated better data to the sponsors.

Note that any one solution may not require the use of all 9 performances listed.  Individuals applying for ID badges will be asked to describe how they demonstrated at least 4:9 performances.

Can you see yourself doing these performances?  Can you see yourself doing at least 4 of these performances with every learning solution?  Can you see other IDs doing these performances, perhaps differently, but still doing them?  If so, you need to consider applying for a learning solutions development credential.  Go to www.tifpi.org/idbadges.

Want a list of all 9 ID standards?   

Would you like to know about the study -- a practice analysis -- that TIfPI Practice Leaders did to generate and validate nine standards, including Addresses Sustainability?  Go to www.tifpi.org/whitepaper_ID.

How Standards Build Workforce Capability

You may have noticed the dramatic news articles about insufficient workforce capacity.  Business wants to have the right people with the right skills working on the right tasks in the right location.  The drama occurs when locally available workforce does not have the right skills and cannot tackle the tasks.   Enter ‘capability building’.  

Capability:  A measure of the ability of an entity (department, organization, person, system) to achieve its objectives, especially in relation to its overall mission. (Businessdictionary.com).

For many, this definition jumpstarts our thinking and moves us directly to ‘provide training’, but training is only one of the tools in the capability market.  Notice the phrases ‘a measure’ and ‘achieve its objectives’.  In order for capability to develop, there must be a way to measure the workforce in relationship to the work (achieving objectives).  Sounds simple.  In some cases it is.  If all you need to measure is the number of widgets produced per shift, both the measurement and the objectives are relatively straightforward.   However, for many businesses, today’s work is very complex.

Instructional Design and Development

Consider the field of instructional design and development.  The work and market are fragmented, diverse, and undifferentiated (see blog, The State of Instructional Design).  In this field, any one assignment may be simple to produce, where the next one may be extremely complex.  Individuals may be required to use specific tools, specific methodologies, specific techniques, and even specific theories.  Others may have a range of such tools, methods, techniques, and theory from which they are expected to select the appropriate ones.  Comparison of work production is nearly impossible.

Only a fraction of the instructional design and development (ID) workforce comes with degrees in the field.  Everyone else layers ID experience on top of their own (non-ID) specialty where that specialty is could be represented by any other workforce field in existence.  Some simple have talent that they hone through experience.  Others have knowledge and practice supplemented with insight and wisdom. 

Many business leaders would prefer to hire the cheapest available talent, which often exists in distant markets and comes with little or no experience or expertise.   Many of the most experienced and talented seek better assignments, living venues, and pay.  Some choose to move out of full-employment into self-employment in order to fulfill their own dreams, while others start there, and still others are forced into self-employment by a market that refuses to hire experienced employees over age 50.  In the meantime, individuals with newly minted degrees in the field find it difficult to prove sufficient experience to be hired.  Once hired, their career path is fuzzy, at best.  

The field of instructional design and development, in particular, is experiencing the pains of a workforce with capability issues. The right workers are not in the right locations with the right skills and where expert practitioners exist they often find it difficult to distinguish their work from that of charlatans with low prices and expert sales techniques.  For more detail on the state of the ID field see the Whitepaper: ID Practice Analysis and Survey Results published by The Institute for Performance Improvement, L3C (TIfPI).

Standards Measure Competence

Standards provide that measure toward which capability development can build.  Standards are the mark of a successful practitioner in any field.   That is, the competent practitioners already practicing in a field use standards that distinguish their work.  These standards transcend practice venues making them customizable for local needs.  In turn, this means that standards can transcend geographic borders, ideological boundaries, language differences, and even variations in tool sets.  Individuals in different practice venues, geographies, cultures, regulatory environments, with different levels of access to materials or equipment can still successfully demonstrate the ability to meet a standard. 

Think of the world of medicine, the techniques, tools, and resources for suturing wounds vary around the world.  However, every healthcare worker around the world is expected to meet common standards in suturing, but meet them using the tools and resources at hand in their part of the world.  Standards like these define the competent members of the field.

Likewise, instructional designers and developers (IDs) need to have a common set of standards to help them build professional competence.  As of 2014, the most common language for IDs is around the use of development models such as ADDIE, SAM, Lean, Six-Sigma, or around theories posited by learning theorists. There are secondary and tertiary languages around tools (Captivate, Articulate, Lectora, etc.) and production processes such as project management and content management. The complexity of variables abound when application of models, theories, tools, content, and projects create unique results with unique parameters.  Under these conditions, it is difficult to compare work.  In fact, the field does not have standards upon which it can compare ID work.

Defining common standards that cross boundaries is not difficult, even though it does require access to the people who know well the work of competent practitioners and can identify competence and cull out the incompetent. With standards defined, it is time to measure.  Those who meet or exceed standards receive a mark of distinction.  Those who do not need to have the opportunity to improve their level of competence through training, effective supervision, and key work tasks that grow their skills.  This is the real power of standards.  When an individual is not meeting standards, skill building begins. The knowledge of which areas need improvement allows one to focus skill-building efforts, demonstrate success, and grow.  

The Public Promise

Where a workforce needs to build public standing, the individuals who succeed at meeting or exceeding standards need to be publicly recognized.  This is partially a personal reward for their expertise.  However, it is more important as an industry marker showing that the industry has tools for recognizing experts and marketing their expertise. 

This is where credentials come into play.  A credential defines the competence of the individual as one who meets standards.  A competent individual is the implied public promise of all credentials.  Most credentials also indicate whether that credential (and associated competence) is a one-time, lifetime award or one that must be maintained and regularly renewed through professional development or reassessment.

In addition, the purveyors of credentials must provide public information describing the methods that they use to define the standards, measure them, track individual’s maintenance of the credential, and ensure that the credential’s standards remain current as work in the field changes over time.  The rigor involved in setting up and managing credentials provides those purveyors with “authority” for backing the credential.  When in doubt, check the source of the credential to be sure that they are actually measuring competence against standards. Authentic purveyors of credentials will be willing to explain the standards used and the measurement and evaluation used.   

A credential is any mark of distinction; a way to identify competent practitioners within a field of shared knowledge, skills, and behaviors.  On the sidebar that describes types of credentials and some of their unique characteristics you may notice that certifications, some degrees, and some accreditations come with “marks”.  A mark is that set of letters used to promote the credentialed individual or organization as one that meets standards.  You may see these marks as initials – CPA, MD, and Ph.D. are common one – or as icons – ISO or UL marks are common.   

About Badges

Badging like gamification has become a buzzword in the learning industry. Many organizations wish to ‘badge’ their employees and students for work related behaviors. Badges have become difficult to assess.  Badges are used across a wide range of credentials and do not match specific types of credentials.  Therefore, two credentials with very different requirements may have very similar looking badges.   

A badge is merely an icon representing the completion of something (e.g., scouting badges, sports patches) or the acquisition of responsibilities and attendant rights (e.g., law enforcement badges, employee badges). 

In the world of credentials, a badge signifies both the completion of something and the acquisition of attendant rights and responsibilities.  However, it becomes the public’s responsibility to determine what was completed and what the badge holder’s rights and responsibilities are.   Then, they must match their own needs with those of the underlying credential. 

Enter badge verification software.  This software allows badge earners to share their iconic badge through social and electronic media (e.g., email, websites).  Clicking on the badge connect the interested public with a website that houses critical information about:

•  The credential,
• The credential holder,
• The organization providing authority to that credential,
• What the credential holder did to acquire the credential,
• What the maintenance requirements are, and
• The credential holder’s status.  

Badges are a symbol (icon) for the credential.  Employers and clients will want to look deeply into the performance evidence required by each credential.  At this time. there are badges available for degrees, certificates, awards, endorsements, and certifications.  The digital badge itself is a marker.  Any two credentials may have similar badges while being very different in the performance requirements needed to achieve the badge.  The value of the badge is in the authority of the credential.  Seek out the authority backing your badges.

Emerging Standards and Microcredentials for IDs

TIfPI has completed a practice analysis that defines nine new standards for instructional designer and developers.  They have defined the standards and the performances expected for each as they relate to learning solution development, a subset of the overall field of instructional design.  Therefore, they are making a series of learning solution development microcredentials with digital badges available to IDs.

The objective is to strengthen the field by providing evidence-based credentials validating that individual IDs have demonstrated their ability to apply international, theory-free, model-free standards in the development of one or more types of learning solutions.  Individuals providing evidence of their ability to meet all nine standards will be awarded a microcredential (with digital badge) for the development of one of 19 learning solution types.  Individuals may acquire as many microcredentials as they wish.  

Individuals receiving microcredentials will be able to assert that two expert instructional designers evaluated their work against standards and that they have met standards.  The ability to show competence increases individuals’ standing within the field, makes it easier for employers to choose competent candidates, and builds professional credibility for the field.  Standards are the key to measuring and evaluating performance, which in turn creates a language of competence and opportunities for continued growth as well as opportunities to build key skills in order to meet standards. 

Watch this blog for more on each of the nine standards for Instructional Design and Development (ID), which state that the competent ID:

•  Addresses sustainability
• Aligns the solution
• Assesses performance (in learning)
• Collaborates and partners
• Elicits performance practice
• Engages learner
• Enhances retention and transfer
• Ensures context sensitivity
• Ensures relevance

To learn more about the 19 learning solution types, standards, available whitepapers, and application process for learning solution development ID Badge, or to join me for one of the free webinars, Overview of ID Badges, provided by TIfPI, go to www.tifpi.wildapricot.org/idbadges.

As always, comments and discussion are appreciated.  Please share your thoughts and insights.

Go to https://tifpi.wildapricot.org/IDBadges

The State of Instructional Design in 2014

You may have seen the serious elearning manifesto.  It opens an important discussion in the instructional design and development world.  It also underlines the chaos that exists in that field; a cohesive field would not need a manifesto that addresses only a fraction of the work within the field – only the e-learning portion, in this case. It also begs the question of why a manifesto is needed and creates a tension between ‘typical’ elearning and ‘serious’ elearning.  This manifesto underscores the fact that field of instructional design and development has charlatans, wannabes, the tired masses, and top-notch professionals – within just the learning portion of the field.

The Instructional Design and Development Workforce Marketplace  

From the market perspective, instructional design and development (ID) is a diverse, fragmented, and undifferentiated market.  This is an international marketplace workforce with a wide variety of skill levels competing against each other for work and recognition. Whether instructional designers and developers work as internal consultants (a.k.a. staff) or as external consultants, they struggle with the fallout from this complex market. 

What is a diverse, fragmented, and undifferentiated market?

Diverse

Instructional designers and developers (IDs) work in every industry from military to social work, from finance work to entertainment, from government to energy, and everything in between.  IDs work for non-profits, military, government, colleges and universities, public schools, every industry every invented, as well as consulting house that serve the world.    They may be one-person self-supporting businesses or they may members of large teams working multi-million dollar projects and every workplace variation in between.  Diversity in workplace creates a huge variance requirements and expectations.

IDs come to the field through two paths – higher education degreed and lateral movers with subject field experience.  While the degreed members are on the rise, the vast majority of the field comes in with native talent and expertise in an industry’s content field.   This highly diverse background experience makes it difficult to compare even entry-level candidates. 

Then, there is the work itself.  Some IDs specialize in one type of solution – just elearning, or just instructor-led, for example.  Others are tasked with creating unique solution sets that address specific needs.  Some are expected to be expert technical writers, while others are expected to be graphic artists and technologists, and some must be everything to everyone.  Some use complex software to create their solutions, with others work with minimal resources in very resource-restrictive environments.  Diverse working environments and expectations create uneven and often unrealistic expectations in employers.

Geography, industry, the size of organization all work to create a very diverse workforce. In addition, this is a creative workforce that often brings the diversity of the creative -- music, art, color, flow, drama, and more. 

Fragmented

US Bureau of Labor and Statistics (BLS) list one role within the Professional and Services sector, the Training and Development Specialist. There is no national labor role for Instructional Designer or Instructional Developer or Instructional Technologist, even though there are degree programs in colleges and universities across the United States and around the world.  However, BLS indicates that the demand for Training and Development Specialists in expect to increase by 15% between 2012 and 2020, adding more than 35,000 new jobs in the United States alone.  That this important government organization does not even recognize the field of instructional design and development creates disenfranchisement, as well as misunderstandings between employers and workers. 

Within the field, there is a certain amount of distrust between instructional designers based on training.  Those with degrees, tend to trust and value IDs with degrees more than those who come to the field without.  The lateral movers with field experience tend to distrust the degreed practitioner who brings academic knowledge of learning theory, but is weak in business acumen.  Since different backgrounds mean that individuals come with different languages and ways to describe their work, the wedge of terminology creates an internal fragmentation.  

Undifferentiated

Check out the job boards for Instructional Designer.  A quick review of job listings will show that most instructional designer job listings are wish lists consisting of a general statement of work asking IDs to be all things to all people – especially, senior leadership.  These job descriptions go on to list a smorgasbord of tools in which the ID must be an expert.  Then, the typical ID job listing is capped off with a need for expertise in a specific development methodology – ADDIE, lean, six-sigma, SAM, etc. – and perhaps even the need to be an expert in the business field, as well.

Add to this, the growth of off-shoring in instructional design and development.  Many employers are willing to choose the cheapest ID resources for their project rather than choosing the ID that best matches their work. 

To this, we can add the fact that there are dozens of names for similar roles – Instructional Designer, Learning Developer, Elearning Developer, Learning Specialist, Learning Developer, Learning Analyst, Learning Architect, Learning Strategist, Education Specialist, and more.  In some cases, there is an implied career progress with position titles mark I, II, III, IV. 

As with every workforce, there are charlatans.  Unless a manager or client is, themselves, and instructional designer, they will find it difficult to distinguish the professional who produces quality work from the charlatan with a good line of schmooze.   This inability to discriminate is the greatest challenge in the industry and increases the fragmentation.

Standards Guide Capability Building

A key to building cohesion, capability, and capacity within any distressed workforce would be defining standards.   The Institute for Performance Improvement, L3C (TIfPI, www.tifpi.org) has just completed a practice analysis of instructional designers.  Watch for the whitepaper, coming soon. 

Out of this analysis comes a set of nine international, theory-free, model-free standards for learning solution development.  Note that these standards focus on development and do not include front-end analysis (needs assessments), delivery, project management, content management, or technology.  Starting with a definition of development standards focuses the field on production standards.  

TIfPI’s instructional design and development experts working with TIfPI’s credentialing experts defined a series of certifications for the learning and solution development portion of the field.  These nineteen certifications are microcredentials – a credential focuses on a subset of the greater field. Whereas a full certification addresses the breadth of the field, microcredentials, often called endorsements, highlight a strength in a specific area.  Today, these credentials have digital icons, called digital badges, which allow credential earners to promote their qualifications through social media.  For more on these credentials see https://tifpi.wildapricot.org/IDBadges.

Coming soon…

Watch this space for more on the emerging international, theory-free, model-free ID standards and access to the practice analysis behind these credentials, or attend the free webinar, Overview of ID Badges.   

Watch for the next in the series -- How Standards Build ID Workforce Capability.

Gamification versus Game-based learning (GBL)

GBL vs Gamification?  I have been building learning games and building games into learning for decades. Both are game-based learning (GBL).  Both, situate the game as a way to experience and learning key skills – usually action-oriented skills, process flow, or decision-making skills.  On the other hand, gamification is actually about the application of game mechanics to a business purpose and is most frequently used in marketing.  Think market loyalty;  www.bunchball.com often has some great white papers on the use of gamification in marketing.   

Can the corporate learning function use gamification in marketing learning?  Certainly!  However, we now find ourselves in a recursive loop where it get hard to tell what’s GBL and what’s gamification.  Try the following examples; they may clarify (or not).  At least this is how I explain it to myself and my clients.

A new employee in corporate orientation is sent on a “treasure hunt” to gather signatures, take pictures, or bring back artifacts from various departments in the organization.  Is this game-based learning or gamification.  Are they learning anything?  Maybe.  They might be learning where to find information, departments, and resources.  To that degree this treasure hunt is a learning game.  I would contend, however, that the real point of the activity is new employee (customer) loyalty and that find information, departments and resources is more about building employee satisfaction than actual skill-building. (Which doesn’t mean that finding stuff is not a skill – just that skill-building is as disguise in this instance for customer/employee satisfaction,)  However, it works and it is a valuable technique.

This same new employee receives an email from the LMS (a coach or other entity) congratulating them on becoming a new member of the organization and reaching the rank of Apprentice Explorer (or any other rank name you want to substitutes).  Our employee receives a badge icon as Apprentice Explorer along with the request that they enroll in and complete the following three e-learning classes to complete the rank of Explorer 1.  (Implied assumption: another badge will come with the new rank.) The courses cover legal requirements, how to use software, working effectively at XYZ  corp, etc.   This is pure gamification of the learning function.  Game mechanics of awards (badges) and leveling up encourage our new employee to complete essential tasks and provides feedback on whether these tasks have been completed.  Again, those tasks include key skills, but the skills are entirely secondary to the purpose of the leveling up based on task accomplishment (a market loyalty technique for return engagements).    

A few months later our new employee attends a workshop on quality assurance and their role in their organization’s interpretation of quality improvement.  The workshop includes a 1-hour game using a board game format to walk learners through the quality process and discover key aspects of that process (what happens when documentation is incomplete, reviews, change initiatives, etc.).   This is pure game-based learning.  The purpose here is build familiarity with the process steps, decisions, and artifacts of quality assurance and to build a common set of expectations for what all employees do as part of corporate quality improvement.  The goal is learning -- building knowledge and skills.  If the learning is fun enough to also build customer (employee) satisfaction, that’s a nice benefit. 

A year or so later, our employee moves up to a somewhat higher level position in the company.  In doing so, they are now expected to participate in a 1-week long goal-based scenario or war game.  For example, our employee moves into a regional sales executive role that requires a 1-week simulation of the sales process.  Herenew sales execs have to build a cold-call list, make a certain number of cold calls,  build a warm-call list (previous clients who are not using a newly released product – a white space sales opportunity), call the warm-client, give a web-based product demo, demonstrate closing the sale, use the CRM software, and write a quarterly report of sales results. This simulation might be done entirely with actors as clients or staff acting as clients.   The goal here is learning and demonstrating skills.  Everyone involved is interested in whether this employee can do key skills and make key decisions.  No one really cares whether the employee has a good time or has fun… or not.  This form of game-based learning is very serious and very intense with high stakes and serious “do you keep the job” implications.  Fun and loyalty are entirely outside the purpose.  However, anyone designing one of these war game/simulations also knows that they need to build in fun, positive feedback, encouragement, and opportunities for learners to feel good about themselves and the company.  They will build in social events, mini-competitions, prizes, team building, and progress markers to help learners power through the very intense and focused learning required by these events.   In the end, the learner comes out feeling very upbeat and positive about the company (employee loyalty) and them themselves and their skill set.  However, the purpose of simulation game was skill learning; employee loyalty was incidental (put in place to keep the employee from dropping out of the intense and often fearsomely competitive learning experience.)  

With these four examples, we can see how gamification and game-based learning blur.  Not sure which you're seeing.  Consider using the primary purpose of the activity as your guide to whether it’s gamification (game mechanics applied to business functions) or game-based learning (games to build skills). Is the purpose building satisfaction and return engagements?  If so, it's probably gamification, even if it is being done in the classroom.  Is the purpose learning and demonstrating skills and knowledge?  If so, it's probably game-based-learning. 

Note:  Some games (e.g., the ubiquitous Jeopardy) are about recall.  The question becomes whether these are valid learning games.  If recall is the skill required, then, yes, these games are game-based learning.  Otherwise, they are gamification of the classroom in order to build learner excitement through fun (a game mechanic) and  to consolidate knowledge before moving on (leveling up – another game mechanic.)  In other words, their purpose is to keep the learning come back into the classroom for the next dose of "learning".   The technique works and can be a valuable, low-cost addition to the learning classroom.  A learning objective that states "name the 9 learning events" is about recall and a game like Jeopardy could be a valid learning game.  However, an objective that states "explain the 9 learning events" might use a Jeopardy-like game to recall the events and build enthusiasm via this classroom gamification technique. 

Comments invited.  Please share your opinions.    

Does {X} Get You Hired?

In the past year to eighteen months I have been a member of many dialogues about the value of something educational – a degree, a certificate, a course, volunteering – in terms of getting the participant hired or promoted.  This seems to be the current hot topic in performance improvement.

… And it implies a line of demarcation and a rite of passage across that line… that is, you in or are you out?

Is that good or bad?  To be determined.  Let's consider the following…

A friend and mentor of mine, Judy Hale of Hale Associates has worked with accreditations and certifications for years and has written one of the field’s go-to books on the topic, Performance-Based Certification: How to design a valid, defensible, cost-effective program.  In recent meeting discussions about emerging certifications at ISPI, she has mentioned meeting with the Department of Labor, which now is requesting (requiring?) a new criterion for their accepted certifications.  The criterion?  Does having this certification make a difference in hiring?  That is, will people with this certification get hired over those without it? 

In one way, this is great progress; while in another way is it is just another kind of a short cut.   Employability as the validation may be just as wonky as the learning objectives that have been dumbed down to little more than a list of topics (but that's another blog). 

Look at this way.  Last year one of my clients was a community college developing programs that they wanted to promoted as certifications, where mostly they were certifying course attendance and passing both knowledge and hands-on tests.  However, their goal was to provide courses to the unemployed.  The courses were the easy part.  The tough part was determining whether any of their programs made a difference in hiring.  Well, some roles are synonymous with their certification (Certified Nursing Assistants, for example) – one must have the training program and certificate in order to get hired for the role.  Besides that program has been around a long time and well validated. 

However, a direct correlation may be more about completing than doing the work. 

The veterinary equivalent, the veterinary assistants, are just now beginning to be certified; in fact, all veterinary assistants in the United States (not just my area) were uncertified as late as 2010 and only a small fraction of one percent certified in 2011.  A new program and certificate may, in fact, be improving the work of the untrained employees who are already doing the work and using a certificate as a stepping stone.  It may also improve the chance of that individuals who are currently unemployeed will get work.  Both are likely.  However, there is no data about the baseline and no way to track progress after completing a program, so how can we measure its effectiveness?  In ten years, the situation will be like the CNA, where you can't get hired without a 6-week certificate program under your belt. 

Is employability the criterion here?  One certification can show a tight correlation between certification and job, while the other cannot merely because it’s too new to have data and has a history of employing the untrained.  Which program is better at preparing the learner for the role? 

Okay, another example.  Last week I received a call from a local journalist who was working on an article about the value of volunteering and the way that businesses were using it to promote  -- themselves both internally with their employees and externally with their clients.  We might consider this an item on the third line of the triple bottom line -- the corporate social responsibility -- with the other two being the top line (revenue) and the bottom line (net profit after taxes and expenses).   Here the social responsibility factors include work-life balance of employees, giving back to the community, receiving awards for work done, receiving publicity for work done, etc.  That employees who participate might keep their job when others are laid off, those who had volunteer work in their resume made it through the hire process or got a promotion -- those are the open questions for this journals. The conversation started around a volunteer project that I had headed up as volunteer with a professional society.  Did it get any of us a job or promotion?  No.  As a professional society, we did get promote the project that created a lot of interest among our members.  However, when members asked about participating in similar projects, they were unwilling to commit the time required.  In the end, the journalist and I talked about the value of volunteering with a professional society as one way that I provide a ‘gut check’ for my clients about whether my certification by that society is valid and valuable.  That is, having a certification might not get me hired but having volunteer work with the organization providing it might validate the certification.   

Let's try a different look at employability measures.  Not too many months ago, the key indicator of getting hired as an instructional designer was whether you had 3-5 years of Flash and Photoshop experience -- another X variable.  For many organizations the requirement to have that experience was obviously less than astute because of the rest of the position description wandered; it wasn’t clear that they had the ability to put someone with 5 years of high-tech instructional design and development experience to work using that experience. 

It’s still an open question.  Does having {X} – degree, certificate, course, volunteer experience, or specific tool experience – make one more employable? Maybe we should also ask whether those individuals with X produce a work with higher quality, quantity, speed and flexibility… and whether that is what the employer really wants.  Our line of defense for these solution sets wobbles with openly visible gaps in the line.

Actually, the most needed question is whether ‘employability’ is the best test of a {X}.  Employability may be only one part of the story and the measure of whether one is “in” or “out” may need to be more complex than do you have {X}? Line between ‘in’ and ‘out’ gets a bit convoluted when we zoom in to look closely at whether we have the right performance measures at all.  

Let's talk.  What examples do you have of validating a program or accreditation?  How effective are the measures in demonstrating the success of that program/accreditation?  

Photographs by Sharon Gander © Jan 2012