November 19, 2013

Working on new "Information Quality" book, to be published in 2015

Ron Kenett and I are working on a new book titled "Information Quality: The Potential of Data and Analytics to Generate Knowledge", to be published by John Wiley & Sons in 2015. The book introduces, conceptually and with examples, the notion of Information Quality (InfoQ), which is the potential of a dataset to achieve a goal of interest, using data analysis. The InfoQ framework is relevant in the design, monitoring and general assessment of work performed by statisticians and analysts who deploy analytic tools on a given dataset, to generate knowledge. The book shows how to bring together the notions of goal, data, analysis and utility that are the main building blocks of data analysis within any domain. Whether the information quality of a dataset is sufficient is of practical importance at many stages of the data analytics journey, from the pre-data collection stage (to determine what data to collect) to the post-data collection and post-analysis stages (evaluating the usefulness of the analysis). It is also critical to various stakeholders: data collection agencies, analysts, data scientists, and management. Designed as a prime reference and textbook on the topic of InfoQ, the book will be useful in traditional data analysis courses (statistics, data mining, data analysis, business analytics, operations research, etc.), and especially as part of the growing number of graduate programs in business analytics and data science. The book will consist of three parts: Part I will introduce InfoQ and the InfoQ framework. Part II will present case studies of real-world analysis using the InfoQ framework, in a wide range of applications (education, healthcare, information technology, risk management, marketing, surveys, and more). Part III will be dedicated to implementation issues of InfoQ.

November 15, 2013

New working paper: "Linear Probability Models and Big Data: The Good, the Bad, and the Ugly"

Linear regression is among the most popular statistical model in social sciences research. Linear probability models (LPMs), which are linear regression models applied to a binary outcome, are commonly used for various reasons, despite criticisms of such usage. We carry out an extensive study to evaluate the use of LPMs in the realm of "Big Data", where large samples and many variables are available. We evaluate performance in terms of coefficient estimation as well as predictive power. We compare performance to alternatives suggested in the literature. We find that the LPM is beneficial for descriptive modeling when the outcome is naturally binary, whereas it is beneficial for predictive modeling when the outcome is binary by discretization. We motivate and illustrate our study through an application to modeling price in online auctions, using real data from the online auction site eBay. The competing title for the paper is "Everything you wanted to know about LPMs and were afraid to ask". To read more, download the SSRN working paper.

October 28, 2013

"To Explain or To Predict?" at Rotterdam School of Management

I'll be giving a talk on "To Explain or To Predict?" at Rotterdam School of Management on Monday, Nov 4, 2013. My host, Otto Koppius, is also my co-author on a related MISQ paper "Predictive Analytics in IS Research". In the talk, I will combine information from this paper and my Statistical Science paper "To Explain or To Predict?" For further details, see this page.

October 26, 2013

Upcoming talk "To Explain or To Predict?" at Darden School of Business

I'll be giving a talk on "To Explain or To Predict?" at UVA's Darden School of Business on Nov 8, 2013. For more information, see this page.

October 9, 2013

Talking about the flipped (MOOC-style) Business Analytics course at INFORMS


I'll be describing and discussing my efforts and experience in re-designing the Business Analytics Using Data Mining course as a hybrid semi-MOOC course in tomorrow's 11am Analytics/INFORMS-Ed session at INFORMS. See here for location and details. I'll also discuss teaching data mining in a b-school in the 8am panel on Teaching Data Mining.