Skip to content

Reestimate preferences from data

At the end of an experiment, your application will compute posterior estimates based on an individual's responses. However, you may want to reestimate preference parameters using experimental data that has already been collected using a different choice model or prior distribution. You can do this using the Python notebook /tools/notebooks/Reestimate_preferences.ipynb that we have provided.

When you run the notebook, you will query your DynamoDB database for your existing experimental data. For each respondent, you calculate new posterior preference estimates based on the new prior distribution (theta_params) and choice model (likelihood_pdf) that you define.

Specify Model Components

You can specify a new prior distribution, new likelihood function, or change the number of samples that are used to calculate the posterior distribution. You should specify the following components:

  • theta_params: This specifies the prior distribution.
  • likelihood_pdf: This specifies the likelihood of observing each answer in answers.
  • size_thetas: The size of the sample drawn from theta_params. Since speed is less important outside of an experiment, you can improve the precision of estimates.

Note that the choice model defined in likelihood_pdf can depend on new preference parameters defined in theta_params.

Alternatively, you can edit all model components in the /app/bace/user_config.py file directly and import them into your notebook using:

from app.bace.user_config import theta_params, likelihood_pdf, size_thetas

Specify Method for Estimating Preferences

By default, Population Monte Carlo (PMC) is used to estimate preference parameters given the design and answer histories for each respondent. If you want to use an alternative method to estimate preferences, such as logistic regression, you can edit the notebook to do so. The portion of code that must be updated is indicated clearly in the notebook.

Results

By default, a .csv file storing the new preference estimates is exported to the same folder and will be ignored by .gitignore. The export location can be changed by editing the output_file variable in the notebook.

For each individual profile_id, the script calculates the mean and median estimates for all preference parameters, indicates the number of designs an individual answered, and indicates whether the reestimation was successful. Additional information can be saved by editing the notebook directly.