Correcting Four Types of Error in Survey Design

Correcting Four Types of Error in Survey Design The world's largest professional association dedicated to the training and development field Login | International | My Profile |Chat |Contact Us | Join or Renew .ui-autocomplete { max-height: 400px; width: 295px; overflow-y: auto; /* prevent horizontal scrollbar */ overflow-x: hidden; } .astd_autocomplete a { border: none!important; font-weight: normal !important; background: #fff!important; color: #333; font-size: 11px; } ul.astd_autocomplete a, .astd_autocomplete ul.astd_autocomplete a:hover { color: #333 !important; } .astd_autocomplete li.category { font-size: 11px; font-weight: bold; padding: 7px 0; margin-bottom: 5px; display: block; text-align: right; margin-top: 10px; } .astd_autocomplete div.inner-border { border-bottom: solid 1px #dedede; } .astd_autocomplete div.inner-label { background: #fff; padding-right: 9px; margin-top: 10px; } .astd_autocomplete a:hover { background: #ededed!important; } Members Publications Conferences Education CPLP Certification Communities of Practice Enterprise Solutions Professional Resources Store Home > Publications > Blogs at ASTD > Learning & Development Blog > Correcting Four Types of Error in Survey Design Correcting Four Types of Error in Survey Design

Wednesday, July 31, 2013 - by Patti P. Phillips

Subscribe to the  newsletter. Congratulations! You are now subscribed to the  newsletter. Subscribe to the  Newsletter First name: Last name: Email Address: div.newsletter-container { border: solid 1px #dedede; background-color: #fafafa; position: relative; float: left; padding: 10px; margin-bottom: 15px; margin-right: 15px; border-radius: 4px; -moz-border-radius: 4px; -webkit-border-radius: 4px; font-family: 'Droid Sans', sans-serif; -webkit-box-shadow: 2px 2px 6px #1c8dda; -moz-box-shadow: 2px 2px 6px #1c8dda; box-shadow: 2px 2px 6px #1c8dda; } div.newsletter-container a, div.newsletter-container:visited, div.newsletter-container a:hover, div.newsletter-container a:active { color: #777; margin-top: 5px; display: block; margin-left: 40px; } div.newsletter-container img.icon { width: 28px; height: 28px; float: left; margin-right: 10px; } div.newsletter-success { color: #468847; background-color: #DFF0D8; border-color: #D6E9C6; } #newsletter-subscribe label { font-family: 'Droid Sans', sans-serif; } input.invalid { border: solid 1px red !important; } Send to Kindle

Survey design is a balance between art and science. Knowing the types of error that affect the quality of a survey and the subsequent results is an important first step in addressing the science side of the scale.

According to Don Dillman and his colleagues (Internet, Mail, and Mixed-Mode Surveys: The Tailor-Made Design Method, Sage Publications, 2008), there are four types of errors we need to consider when designing surveys:

coverage error sampling error response error measurement error.

Coverage error refers to the opportunity we give all potential respondents to respond to the survey. For example, if you want to survey employees to determine their overall perception of the learning function, and 20 percent of the employees work in the field without access to a computer, would you administer your survey using an electronic channel of delivery?

Many would argue Yes! And then ignore those people in the field. But if the opinions of your field people are just as important as those employees with electronic access, you need an alternative approach to delivering the survey to the field. Have you thought of using the internal mail courier, or how about postal service? Recently, we ran across a group that actually faxes surveys to respondents.

Sampling error occurs when the sample is too small to adequately infer survey results to non-respondents. There is a lot of confusion about sampling and when and how to use it. Sampling was devised as a way to collect data from the few to infer to all. But “all” depends on the information you seek.

Your population is defined as the group of people who have the information you want. So, if you want to survey the general population of 50,000 employees about their opinion toward the new smoking ban, then sampling is a way to avoid costs of collecting data from the entire population. A simple tool to help you calculate sample size is located at http://www.raosoft.com/samplesize.html.

But if you want to determine the business impact and ROI of a leadership program for the 25 people who completed the program, your population is the 25 and you administer the survey to the entire population of 25. Your results reflect the results of the 25, but do not suggest the same results for anyone outside that group.

Response rate error is sometimes confused with sampling. Whereas sampling concerns gathering data from a small, representative group from the population, response rate correlates to getting the data back from those to whom you’ve administered the survey. In order to ensure a positive response rate, we suggest you develop a survey administration plan that lists steps you will take before administering the survey, during the data collection period, and after the survey closes that will entice people to respond.

Measurement error is a critical type of error on which we should place laser-like focus. Unfortunately, it is an area on which less attention is paid than response rate. And like it or not, if you have 100 percent response rate on a survey with poorly designed questions, you have nothing. So, measurement error is the first place to focus when designing surveys.

To mitigate measurement error, make sure you ask the right questions, the right way. How do you know you are asking the right questions? Take a look at your objectives. How do you ask the questions the right way? Write questions that allow respondents the opportunity to provide the most accurate responses possible.

Correcting Four Types of Error in Survey Design Patti P. Phillips 2013-07-31

Dr. Patti Phillips has focused on the implementation of ROI evaluation for the past seven years. She assisted in building an international evaluation consulting business, helped launch the ROI Network, and developed and managed international partnerships with organizations interested in implementing the ROI methodology. Prior to her involvement with ROI, she worked 13 years in the electric utility industry. As the manager of market research and planning and marketing administration, Phillips helped launched a number of initiatives, including Marketing University, a unique learning environment for new sales representatives.

Phillips works with organizations to implement accountability processes including ROI. She has been involved in ROI impact studies in a variety of industries around the world, including private and public sector organizations. Along with implementing ROI evaluation, Phillips is the president of the ROI Institute, a research and education organization dedicated to advancing the understanding and practice of ROI evaluation. Her work has allowed her to author and edit a number of publications, including:

The Bottomline on ROI (2002) published by the Center for Effective Performance, which won the 2003 ISPI Award of Excellence The Human Resources Scorecard: Measuring the Return on Investment (2001), published by Butterworth-Heinemann Measuring the Return on Investment, Volume 3 (2001) Measuring ROI in the Public Sector (2002) Managing Evaluation Shortcuts (2001) Mastering ROI (1998) published by ASTD

She has served as contributing author in a number of publications, including Donald Kirkpatrick's Evaluating Training Programs (Berrett-Koehler, 1998), Lorraine L. Ukens's What Smart Trainers Know (Jossey-Bass/Pfeiffer, 2001), and Allison Rossett's The ASTD E-Learning Handbook (McGraw-Hill, 2002). She served as editor of the "Symposium on the Evaluation of Training" in the International Journal of Training and Development 5(4) December 2001.

patti@roiinstitute.net

Communities of Practice:   Learning & Development

.leads_content a, .leads_content a:active, .leads_content a:hover, .leads_content a:visited { color: white; text-decoration: underline; } .leads_content { position: absolute; top: 300px; left: 50px; width: 340px; color: white; padding: 20px; background: #000; background: rgba(0,49,81,0.9); } .leads_content > div { opacity: 1; } .callout { font-size: 15px !important; font-weight: bold !important; }

Authored By: Patti P. Phillips

Dr. Patti Phillips has focused on the implementation of ROI evaluation for the past seven years. She assisted in building an international evaluation consulting business, helped launch the ROI Network, and developed and managed international partnerships with organizations interested in implementing the ROI methodology. Prior to her involvement with ROI, she worked 13 years in the electric utility industry. As the manager of market research and planning and marketing administration, Phillips helped launched a number of initiatives, including Marketing University, a unique learning environment for new sales representatives.

Phillips works with organizations to implement accountability processes including ROI. She has been involved in ROI impact studies in a variety of industries around the world, including private and public sector organizations. Along with implementing ROI evaluation, Phillips is the president of the ROI Institute, a research and education organization dedicated to advancing the understanding and practice of ROI evaluation. Her work has allowed her to author and edit a number of publications, including:

The Bottomline on ROI (2002) published by the Center for Effective Performance, which won the 2003 ISPI Award of Excellence The Human Resources Scorecard: Measuring the Return on Investment (2001), published by Butterworth-Heinemann Measuring the Return on Investment, Volume 3 (2001) Measuring ROI in the Public Sector (2002) Managing Evaluation Shortcuts (2001) Mastering ROI (1998) published by ASTD

She has served as contributing author in a number of publications, including Donald Kirkpatrick's Evaluating Training Programs (Berrett-Koehler, 1998), Lorraine L. Ukens's What Smart Trainers Know (Jossey-Bass/Pfeiffer, 2001), and Allison Rossett's The ASTD E-Learning Handbook (McGraw-Hill, 2002). She served as editor of the "Symposium on the Evaluation of Training" in the International Journal of Training and Development 5(4) December 2001.

Share this page: Tweet EMAIL THIS PAGE googletag.cmd.push(function() {googletag.defineSlot('/14020868/Astd_General_MediumRectangle', [300, 250], 'div-gpt-ad-1330464561576-1').addService(googletag.pubads()).setTargeting("logged_in", "N").setTargeting("is_member", "N").setTargeting("templateid", "Blog - Article").setTargeting("cop", ["LD"]);googletag.enableServices();}); More from Learning & Development Improvisation CLICK Matching the Right Design Strategy to the Right Content Aligning Content to Business Goals Designing Classroom Exercises for Highly Technical Content Matching the Right Design Strategy to the Right Content googletag.cmd.push(function() {googletag.defineSlot('/14020868/Astd_General_MediumRectangle_Bottom', [300, 250], 'div-gpt-ad-1343420926119-0').addService(googletag.pubads()).setTargeting("logged_in", "N").setTargeting("is_member", "N").setTargeting("templateid", "Blog - Article").setTargeting("cop", ["LD"]);googletag.enableServices();}); Filter By Communities of Practice 3

Career Development 1

Global Human Resources Development 1

Human Capital 104

Learning & Development 4

Learning Technologies 1

Sales Enablement 1

Senior Leaders & Executives 1

Workforce Development Filter By Tags 8

Books 6

Conferences 10

Evidence-Based Best Practices 1

Higher Education Filter By Archives 45

20131

August6

July5

June7

May6

April9

March6

February5

January59

20124

December4

November7

October4

September8

August10

July6

June5

May6

April5

March3

20112

March1

February About ASTD Contact Us Terms & Privacy Jobs at ASTD ASTD Job Bank Chapters International Media Contacts Advertise with Us

Stay connected with: 

ASTD is the American Society for Training & Development © 2013 ASTD | All Rights Reserved 1640 King Street, Box 1443 Alexandria, VA 22313-1443, USA
Phone:703.683.8100 Fax:703.683.8103 Munchkin.init('277-BNU-400'); (function (d, s, id) { var js, fjs = d.getElementsByTagName(s)[0]; if (d.getElementById(id)) return; js = d.createElement(s); js.id = id; js.src = "//connect.facebook.net/en_US/all.js#xfbml=1"; fjs.parentNode.insertBefore(js, fjs); }(document, 'script', 'facebook-jssdk'));

View the original article here

Types of Training

Employee Training is more than just an event where trainer talks and trainees listen or take notes, its has far reaching effects on business and its performance both in the short and long term. Training acts as a pathway to learning, Learning drives change and change is key to survival in the fast changing business world. There are different kinds of activities that require training. The most of them are safety, customer service, computer skills, quality initiatives and communication. As mentioned benefits of well-done training include (for both individuals, teams and organisation)

enhanced skills, greater ability to adapt and innovate, better self-management, performance improvement. effectiveness and productivity, more profitability and reduced costs, improved quality.

Training can be designed to meet a number of objectives and can be classified in various ways. As Figure shows, some common groupings include the following:

Required and Regular Training

These are the mandatory trainings that has to be provided due to legal or other business requirement, for instance, fire training, or one given to all employees (e.g., new employee orientation)

Job/technical Training

These training are provided to enhance employee capability to perform the job. (e.g., product knowledge, technical processes and procedures, customer relations)

Developmental and Career Training

Focused on growth of both the individual and the business, these trainings are provided to gear up for the future challenges e.g., business practices, executive development, organizational change, leadership.

Interpersonal and Problem-Solving Training

Aimed at establishment of a conducive, and better working culture, it helps address issure pertaining to both operational and interpersonal problems and aims at improving working relationships for instance interpersonal communication, managerial/supervisory skills, conflict resolution.

Bibliography

Source: Mathis, L, R., and Jackson, H, J., (2010). Human Resource Management. 13th Edition.

View the original article here

Employee Development, Purpose and Types

Employee Development refers to the initiatives that are taken by the individuals and the organizations to enhances the capacity and competence of the individuals that directly builds organizational capability to efficiently and effectively perform their business functions. it has partly from training in business organizations. although the impact and boundaries of training programs are limited. Training is a one time event, although the opinion is that training is a pathway for future development but its a one time activity whereas development is an ongoing process. increasingly there is a shift from training to development, it not only reflects the awareness of learning and its role but also the strategic implications of learned human resource. Primarily Employee; Development serves two objectives.

Brining an improvement in Organizational Performance,Developing the skills, knowledge and attitudes of Employee.

In both cases, learning is central, initiative are taken whereby individuals learn through varied means and mediums.

Both organizational performance and individual growth are realized from the learning capability of the individuals and the organization. The major drivers for the increased emphasis on learning capability include the need for

Continuous ImprovementInnovationContinuous Adaptation.

Continuous improvement, A concept that grew in 1980's with the the implementation of total quality management (TQM). The emphasis in TQM is on measurement of increased performance through constant feedback and reflection. The process is referred to as Single Loop Learning. The intentions is on constant learning to performance better and make business processes more effective and efficient, the work processes are improved through incremental improvements over time.

Newness in service and product delivery is on reason why organizations are able to attain competitive advantage. but doing so is not easy, it requires business to take risk and innovate. Central to this process is being creative and learning constantly from mistakes, actions, micro and macro environment. Learning aids in understanding new ways of reconfiguring concepts and ideas.

Learning and reflection are not one time activities, in order to reap rewards constantly one has to engage in these activities continuously. Retaining competitive advantage requires businesses to constantly learn, adapt to changing market condition and respond to the external market conditions that includes customer needs, new competitors, and new technology.

The emphasis of Employee development is on formal education, job experiences, assessment centers, and mentoring and coaching programs. All of the aforementioned programs are focused on developing competencies directly related to future contribution to an organization as they are future oriented, The aim of the learning initiative shall be on the employee’s current position plus on the preparation of employees for future roles.

Formal Education. University Degree programs or "Corporate Universities" form the Formal education learning function. It involves lectures by business experts/senior executives in the organization or from outside the organization.

Job Experiences. Job Experiences are made up of working through problems, decision making, interpersonal relationships, and task management. Job Experiences aid in development through facing new tasks and challenges. this is done through stretch the current level of skills, through forcing oneself to learn and develop to grow and create its value.

Assessment. The process of Assessment involves collecting information and providing feedback about performance, skills, and behaviors on the job. This needs to be a constant process where by support is also provided on how to develop the skills required and how to rectify the current anomalies. Methods that are used for assessment include instruments/Questionnaires, other forms include personality assessments, psychological tests, and leaderless group discussions.

Mentoring and Coaching Programs. In order to develop employees an effective tool is Mentoring, either one-on-one or with a group, it has been a potent way to develop employees. In mentoring an experienced senior employee who works one-on-one with a less experienced employee (prote´ge´). It is important that for the mentoring process to be effective it has to be based on shared interests, values, or similar personalities. On the other hand Coaching involves a manager or peer who works with an employee to develop skills and provide feedback. Coaching works in a way where coach works one-on-one with employees, by provision of information and resources, that aids in employee learning on there own.

Businesses exists to grow and develop, the same is true for the employees, Organizations that are focused on employee development are better equipped to achieve organizational development. It is important that plans are tailored and modified to appeal to separate individuals and groups, as it would help in making them more efficient and effective.

Bibliography

Ann Gilley, A., Gilley, J., Quatro, S and Dixon, P. (2009). The Praeger Handbook of Human Resource Management. Praeger: USA

View the original article here