\ABS\Auto Blog Samurai\data\Ravinder Tulsiani Training EDGE\astd\Logo.png)
The world's largest professional association dedicated to the training and development field Login | International | My Profile |Chat |Contact Us | Join or Renew .ui-autocomplete { max-height: 400px; width: 295px; overflow-y: auto; /* prevent horizontal scrollbar */ overflow-x: hidden; } .astd_autocomplete a { border: none!important; font-weight: normal !important; background: #fff!important; color: #333; font-size: 11px; } ul.astd_autocomplete a, .astd_autocomplete ul.astd_autocomplete a:hover { color: #333 !important; } .astd_autocomplete li.category { font-size: 11px; font-weight: bold; padding: 7px 0; margin-bottom: 5px; display: block; text-align: right; margin-top: 10px; } .astd_autocomplete div.inner-border { border-bottom: solid 1px #dedede; } .astd_autocomplete div.inner-label { background: #fff; padding-right: 9px; margin-top: 10px; } .astd_autocomplete a:hover { background: #ededed!important; } Members Publications Conferences Education CPLP Certification Communities of Practice Enterprise Solutions Professional Resources Store Home > Publications > Blogs at ASTD > Learning & Development Blog > Correcting Four Types of Error in Survey Design Correcting Four Types of Error in Survey Design Wednesday, July 31, 2013 - by Patti P. Phillips
Subscribe to the newsletter. Congratulations! You are now subscribed to the newsletter. Subscribe to the Newsletter First name: Last name: Email Address: div.newsletter-container { border: solid 1px #dedede; background-color: #fafafa; position: relative; float: left; padding: 10px; margin-bottom: 15px; margin-right: 15px; border-radius: 4px; -moz-border-radius: 4px; -webkit-border-radius: 4px; font-family: 'Droid Sans', sans-serif; -webkit-box-shadow: 2px 2px 6px #1c8dda; -moz-box-shadow: 2px 2px 6px #1c8dda; box-shadow: 2px 2px 6px #1c8dda; } div.newsletter-container a, div.newsletter-container:visited, div.newsletter-container a:hover, div.newsletter-container a:active { color: #777; margin-top: 5px; display: block; margin-left: 40px; } div.newsletter-container img.icon { width: 28px; height: 28px; float: left; margin-right: 10px; } div.newsletter-success { color: #468847; background-color: #DFF0D8; border-color: #D6E9C6; } #newsletter-subscribe label { font-family: 'Droid Sans', sans-serif; } input.invalid { border: solid 1px red !important; } 
Send to Kindle 
Survey design is a balance between art and science. Knowing the types of error that affect the quality of a survey and the subsequent results is an important first step in addressing the science side of the scale.
According to Don Dillman and his colleagues (Internet, Mail, and Mixed-Mode Surveys: The Tailor-Made Design Method, Sage Publications, 2008), there are four types of errors we need to consider when designing surveys:
coverage error sampling error response error measurement error.Coverage error refers to the opportunity we give all potential respondents to respond to the survey. For example, if you want to survey employees to determine their overall perception of the learning function, and 20 percent of the employees work in the field without access to a computer, would you administer your survey using an electronic channel of delivery?
Many would argue Yes! And then ignore those people in the field. But if the opinions of your field people are just as important as those employees with electronic access, you need an alternative approach to delivering the survey to the field. Have you thought of using the internal mail courier, or how about postal service? Recently, we ran across a group that actually faxes surveys to respondents.
Sampling error occurs when the sample is too small to adequately infer survey results to non-respondents. There is a lot of confusion about sampling and when and how to use it. Sampling was devised as a way to collect data from the few to infer to all. But “all” depends on the information you seek.
Your population is defined as the group of people who have the information you want. So, if you want to survey the general population of 50,000 employees about their opinion toward the new smoking ban, then sampling is a way to avoid costs of collecting data from the entire population. A simple tool to help you calculate sample size is located at http://www.raosoft.com/samplesize.html.
But if you want to determine the business impact and ROI of a leadership program for the 25 people who completed the program, your population is the 25 and you administer the survey to the entire population of 25. Your results reflect the results of the 25, but do not suggest the same results for anyone outside that group.
Response rate error is sometimes confused with sampling. Whereas sampling concerns gathering data from a small, representative group from the population, response rate correlates to getting the data back from those to whom you’ve administered the survey. In order to ensure a positive response rate, we suggest you develop a survey administration plan that lists steps you will take before administering the survey, during the data collection period, and after the survey closes that will entice people to respond.
Measurement error is a critical type of error on which we should place laser-like focus. Unfortunately, it is an area on which less attention is paid than response rate. And like it or not, if you have 100 percent response rate on a survey with poorly designed questions, you have nothing. So, measurement error is the first place to focus when designing surveys.
To mitigate measurement error, make sure you ask the right questions, the right way. How do you know you are asking the right questions? Take a look at your objectives. How do you ask the questions the right way? Write questions that allow respondents the opportunity to provide the most accurate responses possible.
Correcting Four Types of Error in Survey Design Patti P. Phillips 2013-07-31
Dr. Patti Phillips has focused on the implementation of ROI evaluation for the past seven years. She assisted in building an international evaluation consulting business, helped launch the ROI Network, and developed and managed international partnerships with organizations interested in implementing the ROI methodology. Prior to her involvement with ROI, she worked 13 years in the electric utility industry. As the manager of market research and planning and marketing administration, Phillips helped launched a number of initiatives, including Marketing University, a unique learning environment for new sales representatives.
Phillips works with organizations to implement accountability processes including ROI. She has been involved in ROI impact studies in a variety of industries around the world, including private and public sector organizations. Along with implementing ROI evaluation, Phillips is the president of the ROI Institute, a research and education organization dedicated to advancing the understanding and practice of ROI evaluation. Her work has allowed her to author and edit a number of publications, including:
The Bottomline on ROI (2002) published by the Center for Effective Performance, which won the 2003 ISPI Award of Excellence The Human Resources Scorecard: Measuring the Return on Investment (2001), published by Butterworth-Heinemann Measuring the Return on Investment, Volume 3 (2001) Measuring ROI in the Public Sector (2002) Managing Evaluation Shortcuts (2001) Mastering ROI (1998) published by ASTDShe has served as contributing author in a number of publications, including Donald Kirkpatrick's Evaluating Training Programs (Berrett-Koehler, 1998), Lorraine L. Ukens's What Smart Trainers Know (Jossey-Bass/Pfeiffer, 2001), and Allison Rossett's The ASTD E-Learning Handbook (McGraw-Hill, 2002). She served as editor of the "Symposium on the Evaluation of Training" in the International Journal of Training and Development 5(4) December 2001.
patti@roiinstitute.net Communities of Practice: Learning & Development
Authored By:
\ABS\Auto Blog Samurai\data\Ravinder Tulsiani Training EDGE\astd\Patti-Phillips.jpg)
Patti P. Phillips Dr. Patti Phillips has focused on the implementation of ROI evaluation for the past seven years. She assisted in building an international evaluation consulting business, helped launch the ROI Network, and developed and managed international partnerships with organizations interested in implementing the ROI methodology. Prior to her involvement with ROI, she worked 13 years in the electric utility industry. As the manager of market research and planning and marketing administration, Phillips helped launched a number of initiatives, including Marketing University, a unique learning environment for new sales representatives.
Phillips works with organizations to implement accountability processes including ROI. She has been involved in ROI impact studies in a variety of industries around the world, including private and public sector organizations. Along with implementing ROI evaluation, Phillips is the president of the ROI Institute, a research and education organization dedicated to advancing the understanding and practice of ROI evaluation. Her work has allowed her to author and edit a number of publications, including:
The Bottomline on ROI (2002) published by the Center for Effective Performance, which won the 2003 ISPI Award of Excellence The Human Resources Scorecard: Measuring the Return on Investment (2001), published by Butterworth-Heinemann Measuring the Return on Investment, Volume 3 (2001) Measuring ROI in the Public Sector (2002) Managing Evaluation Shortcuts (2001) Mastering ROI (1998) published by ASTDShe has served as contributing author in a number of publications, including Donald Kirkpatrick's Evaluating Training Programs (Berrett-Koehler, 1998), Lorraine L. Ukens's What Smart Trainers Know (Jossey-Bass/Pfeiffer, 2001), and Allison Rossett's The ASTD E-Learning Handbook (McGraw-Hill, 2002). She served as editor of the "Symposium on the Evaluation of Training" in the International Journal of Training and Development 5(4) December 2001.
\ABS\Auto Blog Samurai\data\Ravinder Tulsiani Training EDGE\astd\Get-The-Feed.png)
Share this page: Tweet EMAIL THIS PAGE googletag.cmd.push(function() {googletag.defineSlot('/14020868/Astd_General_MediumRectangle', [300, 250], 'div-gpt-ad-1330464561576-1').addService(googletag.pubads()).setTargeting("logged_in", "N").setTargeting("is_member", "N").setTargeting("templateid", "Blog - Article").setTargeting("cop", ["LD"]);googletag.enableServices();}); More from Learning & Development Improvisation CLICK Matching the Right Design Strategy to the Right Content Aligning Content to Business Goals Designing Classroom Exercises for Highly Technical Content Matching the Right Design Strategy to the Right Content googletag.cmd.push(function() {googletag.defineSlot('/14020868/Astd_General_MediumRectangle_Bottom', [300, 250], 'div-gpt-ad-1343420926119-0').addService(googletag.pubads()).setTargeting("logged_in", "N").setTargeting("is_member", "N").setTargeting("templateid", "Blog - Article").setTargeting("cop", ["LD"]);googletag.enableServices();}); \ABS\Auto Blog Samurai\data\Ravinder Tulsiani Training EDGE\astd\communities_icon.png){kind=icon}
Filter By Communities of Practice 3 Career Development 1 Global Human Resources Development 1 Human Capital 104 Learning & Development 4 Learning Technologies 1 Sales Enablement 1 Senior Leaders & Executives 1 Workforce Development \ABS\Auto Blog Samurai\data\Ravinder Tulsiani Training EDGE\astd\tags_icon.png){kind=icon}
Filter By Tags 8 Books 6 Conferences 10 Evidence-Based Best Practices 1 Higher Education \ABS\Auto Blog Samurai\data\Ravinder Tulsiani Training EDGE\astd\archives_icon.png){kind=icon}
Filter By Archives 45 20131 August6 July5 June7 May6 April9 March6 February5 January59 20124 December4 November7 October4 September8 August10 July6 June5 May6 April5 March3 20112 March1 February About ASTD Contact Us Terms & Privacy Jobs at ASTD ASTD Job Bank Chapters International Media Contacts Advertise with Us \ABS\Auto Blog Samurai\data\Ravinder Tulsiani Training EDGE\astd\0713109GooglePlusfootericon.gif){kind=small}
ASTD is the American Society for Training & Development Phone:703.683.8100 Fax:703.683.8103 Munchkin.init('277-BNU-400'); (function (d, s, id) { var js, fjs = d.getElementsByTagName(s)[0]; if (d.getElementById(id)) return; js = d.createElement(s); js.id = id; js.src = "//connect.facebook.net/en_US/all.js#xfbml=1"; fjs.parentNode.insertBefore(js, fjs); }(document, 'script', 'facebook-jssdk'));
No comments:
Post a Comment