7 CREDIT CHARACTERISTICS, CREDIT ENGAGEMENT TOOLS, AND FINANCIAL WELL-BEING
2. Methods and Data
For this project, Credit Karma administered a voluntary survey to some of its members between
September 7, 2017, and October 31, 2017. During the field period, Credit Karma invited
members to participate in the survey when they logged out of their Credit Karma session by
asking, “Before you go, as part of a joint project between Credit Karma and the CFPB, would you
be willing to take a short survey?” Invitations were presented using a quota sampling scheme
13
13
In this quota sampling scheme, Credit Karma members were segmented into mutually exclusive subgroups defined
by age and credit score and a specified number of responses were sought from each subgroup. The number of
responses sought was set to reflect the distribution of Credit Karma members over the age and credit score groups.
For example, 2 percent of Credit Karma members are between the age of 18 and 24 and have a credit score below
700, so the quota for this group was set at 100 (2 percent of 5000). After the quota for a subgroup was reached
during the data collection phase, no more invitations were extended to members of that subgroup.
targeting 5,000 total responses. No member was invited to participate more than once. Credit
Karma users who agreed to take the survey were directed to a Credit Karma/CFPB-branded
page. The survey consisted of the full 10-question version of the CFPB’s Financial Well-Being
Scale. The 10 questions are included as Appendix A. The survey was administered to 4,559
consumers who accepted the invitation, of whom 4,067 provided complete answers. The survey
data also contain the start and end time and date of the survey.
Each respondent’s survey data were then merged by Credit Karma with their background, credit
report, and website usage data. As part of standard business practice, Credit Karma collects
background information on its members, including when they signed up for Credit Karma
services, their state of residence, and their age. In addition, Credit Karma receives credit report
information at every login, conditional on there being at least a one week delay since a prior
credit report refresh. As a result, Credit Karma received up-to-date credit report information,
including credit score, at the time the survey was administered and could match this
information to the survey responses. VantageScore 3.0 credit scores were available at the time of
the survey and each month going back to January 2017 depending upon the length of time that
an individual respondent had been a registered user of Credit Karma. An additional 252 credit
report variables provide information on utilization, performance, and inquiries covering all
major consumer financial product categories (mortgages, credit cards, student loans, auto loans,
and installment loans).
14
14
The credit report information provided is at most seven days old at the time the survey is taken. A consumer’s
credit report is updated upon log-in unless the existing credit report information is less than seven days old. See
https://help.creditkarma.com/hc/en-ca/articles/115004683866-How-often-does-my-credit-report-information-
update-
In addition, several variables provide summary measures across
several categories (e.g., total number of accounts, number of accounts opened in last 6 months).