DTU Ph.D.-course (02930)

Analysis of Sensory and Consumer Data

at the

Technical University of Denmark,

Lyngby, Denmark

The course will NOT run in 2017 and there are currently no plan for when or if the course will run again

Organized by:

Per Bruun Brockhoff, DTU Compute

Technical University of Denmark

Material and lecture podcasts from the 2015 version are available below!!

If you would like to pursue some of this further:

• Get support for the planning and analysis of your experiments
• Get dedicated courses
• Co-finance collaborative research projects e.g. Ph.D projects
• Get support for dedicated implementations of some of the methodology - e.g. by personalized R Shiny web applications meeting your needs
• Etc

Don't hesitate to contact us:
Per Bruun Brockhoff , Email: perbb@dtu.dk

Our Statistics and Data Analysis section here at DTU Compute has a large experience and interest in entering collaborative contracts - as well pure research contracts as research based consultancy contracts

2015 Material overview and access: (See podcast link table below - OR go directly to raw podcast channel )

Block	Topic	Slides	R and Data	Reading Material	Exercises	Solutions
Mon1	Intro to R Discrimination basic, part 1	SoftwareandRIntroHA.pdf sensR_part1.pdf	sensR_part1_examples.R	eNote1fromCourse02402 (with R intro) sensR_intro.pdf Chapter 7 from the book by Næs, Brockhoff and Tomic. sensR_Refmanual.pdf	sensR_part1_exercises.pdf	sensR_part1_exercises_AND_solutions.pdf
Mon2	Discrimination basic, part2 Similarity testing Replicated data	sensR_part2HA.pdf	sensR_part2.R	The statistical power of Replications, FQP 2003 sensR Vignette on Methodology The Design of Replicated Difference Tests, Meyners & Brockhoff (2003).	sensR_part2_exercises.pdf	sensR_part2_solutions.R
Tue1	Intro to mixed models Basic ANOVA in Panelcheck	IntroMixedModel_Presentation_PBBHA.pdf From_simple_to_3way_ANOVAHA.pdf	introexample.csv TV2.csv	eNote-1 from Course 02429: Introduction to mixed models Selections from the book by Næs, Brockhoff and Tomic.	mixedmodelsinPanelcheck_exercises.pdf	mixedmodelsinPanelcheck_exercises_and_solutions.pdf
Tue2	Mixed models in Consumercheck SensMixed, part 1	SensMixed_ConsumerCheck_2015.pdf	Ham_consumer_attributes.txt Ham_consumer_liking.txt Ham_design.txt	Ph.D. thesis, Kuznetsova, A.(2015) FQP Paper draft on the SensMixed package. (Kuznetsova et al) JSS paper draft on ConsumerCheck (Tomic et al) tutorialSensMixed.pdf	Exercises_Tuesday_afternoon.pdf	Exercises_Tuesday_solutions.pdf
Wed1	PCA and Preference mapping in ConsumerCheck	ConsumerCheck_PCA_2015.pdf	Data_1_liking.xlsx Data_1_QDA.xlsx	JSS paper draft on ConsumerCheck (Tomic et al) eNote2 from Course 27411 - PCA intro with R	Exercises_Wednesday_morning.pdf	Exercises_Wednesday_morning_solutions.pdf
Wed2	PCA, Tucker1 and panel performance in PanelCheck	PanelCheck_August2015HA.pdf	TV2.xls	The book by Næs, Brockhoff and Tomic.	Panelcheck_exercises.pdf
Thu1	Same-difference testing Anot-A testing	AnotA_HA.pdf	examples_thursday_m.R funs.R	Samediffpaper, FQP 2009. Thurstonian models as GLMs, FQP paper, 2010	exercises_anota.pdf	exercises_anota_solutions.pdf
Thu2	d-prime ANOVA Likelihood methods Ordinal data analysis	likelihood_HA.pdf dprime_anova_HA.pdf ordinal_part1and2HA.pdf	examples_thursday_likelihood.R examples_thursday_anova.R ordinal_part1and2.R	Statistical and Thurstonian models for the A-not A protocol with and without sureness, FQP 2011 Analysing Sensory ratings by CLMs. Christensen and Brockhoff, 2013 sensR Vignette: 2ACexamples ordinal: Reference Manual ordinal Vignette: Methodology ordinal Vignette: tutorial on clm ordinal Vignette: tutorial on clmm2	exercises_thursday_anova.pdf	exercises_thursday_anova_solutions.pdf
Fri1	The MAM (scaling correction) d-tilde plotting in ANOVA Sensmixed, part 2	MAMintroHA.pdf Fs_and_dprime_presHA.pdf	SensMixedTutorial.R	tutorialSensMixed.pdf d-tilde plotting paper, Brockhoff et al (2015) Ph.D. thesis, Kuznetsova, A.(2015) Automated Mixed ANOVA, FQP 2015, Kuznetsova et al. The MAM paper, Brockhoff et al (2015) The MAMCAP paper, Peltier et al (2014) The Original Assessor model paper, Brockhoff (1994) FQP Paper draft on the SensMixed package. (Kuznetsova et al) tutorialSensMixed.pdf	Exercises_Friday_morning.pdf
Fri2	The lmerTest package	lmerTest_summerschool_2015.pdf	lecture_lmerTest_Rcode.R lmerTestTutorial.R	Ph.D. thesis, Kuznetsova, A.(2015) lmerTest Reference Manual lmerTestTutorial.pdf JSS paper on the lmerTest package, Kuznetsova et al(2015).	Excercises_Friday_afternoon.pdf

2015 Podcast link table:

Block	Topic	Podcasts
Mon1	Intro to R Discrimination basic, part 1	Course software-an overview - Including a small R intro (13min) The sensR package - An overview of package features (15min) Basic sensory difference testing - And the first use of sensR (23min) Five basic discrimination protocols - And recommendations for protocol choice (7min) Proportion discriminators interpretation - And using sensR to jump between observed and discriminator interpretation (13min) The Thurstonian model - d-primes, resolving Gridgeman's paradox and how to get it from sensR (23min) What is the interpretation of the d-prime? As opposed to just looking at the p-value (4.5min) Power and sample size for discrimination testing - And how to find these with sensR (26min) Power and the five discrimination test protocols - And "first" versus "stable exact" sample sizes (7min)
Mon2	Discrimination basic, part2 Similarity testing Replicated data	Similarity testing - And how to do it with sensR (23min) Power and sample size for similarity testing - And how to do find these with sensR (10min) Replicated difference testing - What is the challenge? Simulations and analysis by corrected beta-binomial with sensR (29min)
Tue1	Intro to mixed models Basic ANOVA in Panelcheck	Mixed model software by the DTU Sensometrics group - An brief overview of PanelCheck, ConsumerCheck, lmerTest, SensMixed, sensR and ordinal (14min) Introduction to mixed models - Motivating mixed models and explaining what they can do for you (14min) The three ANOVA options in PanelCheck - A brief overview (16min) The basic single observation 2-way ANOVA - And the link to paired t-testing (12min) Two-way ANOVA with replications - Panelist-by-product interaction and the concept of a random effect (23min) The proper product difference F-test - And the relation to the standard fixed effect ANOVA table (2min) ANOVA with PanelCheck - A brief tutorial (9min) The 3-way mixed model ANOVA in PaneclCheck - Taking also session and/or product batch effects into account (10min)
Tue2	Mixed models in Consumercheck SensMixed, part 1	SensMixed: The Shiny version. A tutorial for how to analyze more generally structured multi-attribute sensory data by the Shiny browser interface of the SensMixed package (29min) Conjoint analysis in ConsumerCheck - A tutorial (20min)
Wed1	PCA and Preference mapping in ConsumerCheck	PCA and Preference mapping in ConsumerCheck (26min)
Wed2	PCA, Tucker1 and panel performance in PanelCheck	PanelCheck: What is it and what can you use it for - An overview (13min) PanelCheck: Data structure and data import - And what about missing values? (4min) PanelCheck: Workflow - How to use the various features of PanelCheck in your work with sensory panels (7min) PanelCheck: Tucker 1 plots - How to investigate multivariate (dis)agreement between assessors (14min) PanelCheck: Manhattan plots - Visualizing attribute wise explained variances from individually assessor-wise performed PCAs (15min) PanelCheck: Export of plots - Very brief (1min) PanelCheck: Plot based one individual one-way ANOVAs - F plots, MSE plots and p*MSE plots (10min) PanelCheck: Statis - A few words: Weighted PCA based on agreement weights (7min)
Thu1	Same-difference testing Anot-A testing	The A-not-A test - Procedure, Thurstonian model and sensR-tutorial (7min) The same-different test - Procedure, Thurstonian model and sensR-tutorial (20min) Measures of sensitivity - AUC (area under curve), overlap and d-primes. And how to with sensR and ordinal (12min) The A-not-A with sureness protocol - And the soup data example and a little ordinal tutorial (23min)
Thu2	d-prime ANOVA Likelihood methods Ordinal data analysis	Likelihood based confidence intervals - Motivation and examples (11min) Likelihood based confidence intervals - The likelihood function and principle. (11min) Likelihood based confidence intervals - Examples revisited (2min) Likelihood based confidence intervals - Perspectives and more discussion based on sensR output (13min) One-way ANOVA for d-primes - Principles, maximum likelihood and sensR tutorial. (16min) Common multi-protocol d-prime - Estimating and testing the common d-prime (discrimination and/or similarity) (3min) ANOVA for d-primes - Post-hoc tests and comparisons (21min) The ordinal package - An overview of different types of ordinal data (15min) Ordinal data analysis - The cumulative link model and the clm function of the ordinal package (8min) Ordinal data analysis - Extensions of the clm-model, e.g. including differences of product variances (7min) Ordinal data analysis - Mixed model extensions of clms (4min) Ordinal data analysis - A few short examples (8min)
Fri1	The MAM (scaling correction) d-tilde plotting in ANOVA Sensmixed, part 2	MAM: Scaling correction of sensory data - Motivation, background and Sensobase meta investigation (26min) MAM: Scaling correction of sensory data - Basis versus covariate version. Conditional and adjusted versions (4min) MAM: Scaling correction of sensory data - What can R (SensMixed) do for you - including MAM-CAP table information (13min) d-prime like interpretations of mixed model output - An improved alternative to multi-attribute F-barplots (23min) MAM and d-prime in SensMixed - presentation on how to correct for the scaling effect and plot d-prime plots in SensMixed (22min)
Fri2	The lmerTest package

Course Objectives

To improve the ability of analysing human perception data. Some of the newest statistical methodologies will be covered using the open source software R. The packages sensR, ordinal, SensMixed and lmerTest will be used together with the free software packages PanelCheck and ConsumerCheck.

Learning Objectives

A student who has met the objectives of the course will be able to:

· Work with R, Panelcheck and ConsumerCheck

· Plan and analyze simple discrimination and similarity experiments using sensR

· Analyze replicated discrimination data

· Perform and understand simple Thurstonian modeling

· Use mixed models for sensory profile data and consumer preference data by PanelCheck and ConsumerCheck

· Analyze sensory profile data with the newest scale correction method (using R)

· Use PanelCheck and ConsumerCheck for simple analysis as well as multivariate analysis (including Tucker-1)

· Use the R-packages lmerTest and SensMixed to analyze non-standard data with mixed models

· Analyze ordinal human perception data using the R-package ordinal

· Visualize ANOVA results by the newest delta-tilde method using the SensMixed R shiny application

· Know about the newest sensometrics research going on at the DTU Sensometrics group

Participants

The course is aimed at Ph.D-students within non-statistical areas such as sensory science, food science, marketing etc. with interest in data analysis and statistics. It is also well suited for sensory practitioners from industry and scientific institutions or for Ph.D. students within Statistics/Data analysis with interest in human perception data.

Language

All lectures and activities will be given in English.

Organizers

Professor Per Bruun Brockhoff, B324, R220, (+45) 2044 1711, perbb@dtu.dk

Programme, overview

· Monday: Basic discrimination testing using R (package sensR)

· Tuesday: Basic mixed models using PanelCheck, ConsumerCheck and R (package SensMixed).

· Wednesday: Multivariate Analysis using PanelCheck and ConsumerCheck

· Thursday: More advanced discrimination testing using R (replicates etc)(packages sensR and ordinal)

· Friday: More Advanced mixed models using R-package lmerTest (and lme4)

We begin Monday morning at 9am and finish Friday at 3pm.

Teachers:

Professor Per Bruun Brockhoff

DTU Compute, Technical University of Denmark

Study Material:

1. Brockhoff (2011), Sensometrics. In: International Encyclopedia of Statistical Science, Lovric, Miodrag (Ed.), Springer.

2. Brockhoff P. B., Amorim I., Kuznetsova A., Søren Bech, Lima R. R. (2016). Delta-tilde interpretation of standard linear mixed model results, Food Quality and Preference, Vol 49, 129-139.

3. T. Næs, P.B. Brockhoff and O. Tomic, (2010). Statistics for Sensory and Consumer Science, John Wiley & Sons.

4. Brockhoff, P.B. and Christensen, R.H.B. (2010). Thurstonian models for sensory discrimination tests as generalized linear models. FQP, 21(3), 330-338.

5. Brockhoff, P. B., Schlich, P., & Skovgaard, I. (2015). Taking individual scaling differences into account by analyzing profile data with the Mixed Assessor Model. FQP, 39, 156-166.

6. Christensen, R. H. B., & Brockhoff, P. B. (2013). Analysis of sensory ratings data with cumulative link models. J. Soc. Fr. Stat. & Rev. Stat. App., 154(3), 58-79.

7. Christensen, R.H.B. Cleaver, G. and Brockhoff, P.B. (2011). Statistical and Thurstonian models for the A-not A protocol with and without sureness, FQP 22(6), 542-54.

8. Kuznetsova, A., Christensen, R.H.B., Bavay, C. and Brockhoff, P.B. (2015). Automated mixed ANOVA modeling of sensory and consumer data. FQP 40 (2015) 31-38

We will supplement with a number of guides/tutorials and newest papers.

Social activity

Coffee for coffee breaks, lunches and one dinner at a restaurant in Copenhagen is provided for everyone.

Housing

You can find information on Visit Copenhagen's website.

Laptops

Students are required to bring their own laptops for the computer exercises.

R software

We will use the software R throughout the course, and we ask all participants to install R on their laptop in advance of the course. We also ask that you install the graphical user interface (GUI) / Integrated development environment (IDE) RStudio. If you are a confident R user and acustumed to a different GUI, feel free to use that one (though we strongly discourage the standard GUI that comes with R). Both R and RStudio are open source and free of charge.

To install R go to http://cran.r-project.org/ and follow the installation instructions.

To install RStudio go to http://www.rstudio.com/products/rstudio/download/ and choose according to your operating system.

For further assistance, see the following short intro (with relevant links) to get started with R and RStudio:

R introduction - get started (incl. Rstudio). (New 2015 )(13 min)

If you want to watch a few very short videos on how to get started with
R then you can go to this Youtube Playlist:

http://www.youtube.com/playlist?list=PLOU2XLYxmsIK9qQfztXeybpHvru-TrqAP
(A number of short videos illustrating the use of R - shown within a Mac version,
but apart from the appearance of the sub-windows exactly the same works for RStudio
in any operating system)

Basic Statistics Brush-up??

We will assume that you have a background with (at least) an introductory statistics class. If p-values or the idea of confidence intervals etc is a bit far away for you, you can find topic wise categorized links to podcasts on all topics of an introductory statistics class here:

http://introstat.compute.dtu.dk/podcast/modulized-and-ordered-uk-podcast-links/

You can also find a complete online textbook on statistics:

http://introstat.compute.dtu.dk/enote/

More advanced statistics
In the course we will be using the concepts of Mixed Models and also Principal Component Analysis. We do not assume that you are educated in these topics – so we will teach you as we go along. But having some idea of what these things are in advance could probably increase your outcome of the course:

Mixed models:

At the website of DTU course 02429:

http://02429.compute.dtu.dk/

you will find a complete online textbook on applied linear mixed models, and also a complete collection of small podcasts taking you through the course. You will find some introductory lectures there, that you might spend some minutes viewing.

Chemometrics:

At the website of DTU course 27411:

http://27411.compute.dtu.dk/

you will find some podcasts on e.g. Principal Component Analysis.

Looking very much forward to welcoming you in Wonderful Copenhagen!