Silviase/jgov_v1.4_full_ann huggingface dataset:
### 1. Introduction

Growth of the service sector, which accounts for about 70% of Japan's gross domestic product and employment, is essential for the sustainable growth of its economy. In particular, productivity increases through improvements in service quality will need to play a stronger role than ever, since the decline in labor input due to the persistent population decline is expected to be a persistent drag on the economy.

Against this background, the aim of this paper, taking retail services as an example, is to examine the contribution of service quality to service sector output in Japan in order to derive lessons regarding the service sector's role as a driver of macroeconomic growth. Specifically, we examine the following two questions: (1) how much of the variation in retail sales across retail firms is due to differences in the quality of services, including the product variety offered; and (2) how much the variation in real output of the retail sector would change if it were deflated by price indices incorporating differences in quality instead of conventional price indices.

However, due to the economic characteristics of services and the lack of appropriate data, there are methodological difficulties in quantitatively assessing service quality and its contribution to output. First, certain qualitative aspects of retail services cannot be measured using objective indicators, since they are essentially subjective in nature. $ ^{1} $ Examples are the friendliness of sales staff and the sophistication of shop displays. Second, the intangibility and simultaneity of production and consumption of retail services make it difficult for outsiders to grasp the quality of services provided. For instance, precise explanations of a product by sales staff cannot be measured in the same way as the user interface of electronic devices is tested in a laboratory. Moreover, it is difficult to grasp the quality of a service independent from that of the product itself, since consumers' purchase decisions may be affected by both factors. Third, partly due to the above characteristics, detailed information on consumers' purchase behavior at each retail store that would be needed to obtain consumers' assessment of service quality is usually unavailable. Even relatively basic information
that links the products a retailer supplies with the prices and quantities at which they are sold is unavailable. $ ^{2} $

This study attempts to overcome these limitations by relying on three closely related methodological devices. First, we use a massive dataset of barcode-level purchase records at individual retail firms. Specifically, we extract combinations of price and quantity information for a quarterly average of 157,147 products purchased at 1,517 retail firms from a large-scale dataset collected through barcode scans by about 50,000 consumers.

Second, we construct a structural model of consumer demand for the real output of each retail firm to estimate structural parameters capturing consumers' implicit valuation of service quality employing the massive dataset of barcode-level transactions. The theoretical framework in this respect is largely based on Hottman et al. (2016) and Hottman (2016). Hottman et al. (2016) develop a three-level nested constant elasticity of substitution (CES) utility system and decompose the firm-size distribution of heterogeneous multiproduct manufacturers into the contribution of cost, appeal, markup, and product scope. Hottman (2016) uses a similar framework to examine the mechanisms through which the retail sector in the United States affects the attractiveness of cities and consumer welfare. We extend their three-level nested CES utility system to investigate the variation in sales and real output of Japanese retail firms.

However, as will be explained later, if we were to directly employ the approach developed by Hottman et al. (2016) and Hottman (2016) we would not be able to assess the contribution of service quality independently from product quality. Therefore, as the third element of our approach, we introduce a benchmark product and a benchmark product group to normalize the quality parameters of each product and product group across retail firms, which makes a direct comparison of service quality of the same product sold at different retail firms possible and enables us to extract firm-level service quality, which homogeneously affects the sales of all products at a specific retail firm.
The remainder of the study is organized as follows. Section 2 explains the data we use and provides some descriptive statistics. Section 3 presents our structural model of the consumer problem and the retail firm problem. Section 4 then explains the estimation methods. Through structural estimations, we obtain unobserved service quality, unobserved markups, marginal costs, and the elasticity of substitution. Next, Section 5 presents the estimation results and examines the contribution of service quality to variations in sales and real output. Finally, Section 6 concludes.

### 2. Data

Our data source is INTAGE SCI, which is a massive dataset of purchase records of about 50,000 respondents. $ ^{3} $ Respondents scan the barcode (Japan Article Number, JAN $ ^{4} $ ) of products using a portable barcode scanner or smartphone at the time they purchase the item and enter the purchase details (where and when they bought the item, how much it cost, how much they spent shopping in total) by the end of the day. There are more than 10 million purchase records per quarter. $ ^{5} $ The products fall into 14 broad categories and 302 product groups based on the classification of the Japan Item Code File Service (JICFS). $ ^{6} $ The major categories covered are staple food, processed food, beverages, household goods, cosmetics, and drugs. Each purchase record contains information on the retail store belonging to one of 1,517 retail firms at which the product was sold. $ ^{7} $ We classify retail firms into seven retail categories: supermarkets, convenience stores, 99 or 100 Yen stores, hardware/discount stores, pharmacies/drugstores, liquor stores, and department stores.
Table 1 : Summary Statistics

Mean	Median	Std. dev.	10th percentile	90th percentile	Max. value
Firm sales (1,000 Yen)	10,089,385	5,203,766	16,789,878	966,293	23,821,732	162,056,106
Product-group sales per firm (1,000 Yen)	45,211	27,974	61,573	7,459	99,469	5,262,860
Product sales per retail firm (1,000 Yen)	1,941	1,666	1,184	850	3,419	6,778
No. of product groups per firm	174	175	54	103	243	284
No. of products per firm	3,037	2,037	54	618	6,365	26,519

Note: Weighted by the sales of each retail category in each quarter.

Based on the original dataset, we construct a nationwide quarterly database from the second quarter of 2010 to the fourth quarter of 2014 containing the quarterly average prices and total quantities of each barcode-level item sold at each retail firm. Note that, as explained in the following section, we screen the original dataset and limit it to firms that sold the benchmark product of the benchmark product group in each quarter, which we use to compare the quality parameters of each product and product-group across retail firms. As a result, the database on average covers 157,147 different products sold per quarter. $ ^{8} $ Table 1 provides summary statistics of our database. We weight the data by the sales of each of the seven retail categories mentioned above in each quarter and average across quarters. As one can see in Table 1, there are large variations in retail firms' sales. The top firm sells over thirty times as much as the median firm and nearly 170 times as much as the firm at the 10th percentile.

Next, Table 2 shows the distribution of retail firms' sales by quintile. The table indicates that retail firms in the top quintile of sales account for about 70% of total sales. On average, the firms in the top quintile have about 19 times more products than firms in the bottom quintile, indicating that product variety may affect sales variation across retail firms to a certain extent. Table 3 provides a more detailed description of this firm heterogeneity by focusing on the ten largest firms in each of the seven retail categories (where we weight the averages by the sales of the category). The table shows that even the largest firms are not close to being monopolists. On average, the largest firms have a market share of 5.4%, and around one-fifth of all of the sales of firms in the top quintile accrue to the ten largest firms.
Table 2 : Size Distributions by Quintile

Ranked quintile	Quintile market share	Average firm market share	Average firm sales (1,000 yen)	Average no. of product groups per firm	Mean no. of products per firm
5	70.79%	6.21%	29,995,824	201	7,623
4	16.20%	3.34%	12,129,724	176	2,998
3	7.97%	1.92%	6,803,100	189	1,727
2	3.60%	0.70%	2,607,222	193	1,043
1	1.44%	0.34%	1,208,336	116	402

Note: Weighted by the sales of each retail category in each quarter.

Table 3 : Size Distribution by Firm Rank

Firm rank	Firm market share	Firm sales (1,000 yen)	No. of product groups per firm	No. of products per firm
1	5.4%	162,056,106	283	26,336
2	2.7%	82,164,779	273	19,157
3	2.2%	64,941,582	274	13,677
4	1.8%	53,393,314	270	13,465
5	1.4%	41,820,093	271	14,752
6	1.3%	37,985,449	260	10,063
7	1.2%	35,452,648	267	12,076
8	1.1%	32,492,155	256	11,722
9	1.0%	28,987,622	239	9,715
10	0.9%	27,022,369	261	10,711

Note: Weighted by the sales of each retail category in each quarter.

### 3. Model

### 3.1 Preferences

#### 3.1.1 Utility function

We consider a nested constant elasticity of substitution (CES) utility system that allows the elasticity of substitutions and quality parameters of the three levels, i.e., firms, product groups, and products, to differ from each other. Utility $ U_{t} $ at time t is given by the following CES function:

$$ U_{t}{=}\Big[\sum_{r\in R_{t}}(\varphi_{r t}C_{r t})^{\frac{\sigma_{R}-1}{\sigma_{R}}}\Big]^{\frac{\sigma_{R}}{\sigma_{R}-1}}, $$

where $ C_{rt} $ denotes the subutility derived from the real consumption of the product groups sold by retail firm r; $ R_{t} $ is the set of retail firms at time t; $ \varphi_{rt} $ is the perceived overall service quality of retail firm r; and $ \sigma_{R} $ is the elasticity of substitution between retail firms.
Below the utility described above, we assume two CES nests for the product-group level and the barcode level. For the first nest, the subutility from the real consumption of the product groups supplied by retail firm r, $ C_{rt} $ , is given as a CES consumption index of product groups:

$$ C_{r t}=\left[\sum_{g\in G_{r t}}(\varphi_{g r t}C_{g r t})^{\frac{\sigma_{G}-1}{\sigma_{G}}}\right]^{\frac{\sigma_{G}}{\sigma_{G}-1}}, $$

where $ C_{grt} $ denotes the subutility derived from the real consumption of product group g; $ G_{rt} $ is the set of product groups sold at retail firm r; $ \varphi_{grt} $ is the perceived quality of, or the consumer's taste for, product group g at retail firm r; and $ \sigma_{G} $ is the elasticity of substitution between product groups. In other words, $ C_{rt} $ is a composite good composed of the various product groups supplied by retail firm r, adjusted by the substitutability between product groups, where each product group is weighted by its perceived quality.

For the third nest, the subutility from the real consumption of product group g supplied by retail firm r is given as a CES consumption index of products:

$$ C_{g r t}=\left[\sum_{u\in U_{g r t}}(\varphi_{u r t}C_{u r t})^{\frac{\sigma_{U g}-1}{\sigma_{U g}}}\right]^{\frac{\sigma_{U g}}{\sigma_{U g}-1}}, $$

where $ C_{urt} $ denotes the real consumption of product u; $ U_{grt} $ is the set of products within product group g supplied by r; $ \varphi_{urt} $ is the perceived quality of, or the consumer's taste for, product u at retail firm r; and $ \sigma_{U_{g}} $ is the elasticity of substitution between products within product group g. Thus, $ C_{grt} $ is a composite good of the various products within product group g supplied by retail firm r.

We assume that the elasticity of substitution at each level, $ \sigma_{R} $ , $ \sigma_{G} $ , and $ \sigma_{Ug} $ , is constant over time, and $ \sigma_{G} $ , and $ \sigma_{Ug} $ are constant across retail firms.

#### 3.1.2 Consumer problem

We assume a representative consumer and solve for his budgeting decisions via backward induction. In the lowest tier of demand, the representative consumer allocates expenditure across products in a given product group at a particular retail firm. The share of consumer spending on products with barcode u in product group g at retail firm r is given by

$$ S_{urt}=\frac{\left(\frac{P_{urt}}{\varphi_{urt}}\right)^{1-\sigma_{Ug}}}{\sum_{v\in U_{grt}}\left(\frac{P_{vrt}}{\varphi_{vrt}}\right)^{1-\sigma_{Ug}}},\qquad\sigma_{Ug}>0,\varphi_{vrt}>0, $$
where $ P_{urt} $ is the retail price of product u at retail firm r. The corresponding price index for product group g, $ P_{grt} $ , is then given by

$$ P_{g r t}=\left[\sum_{u\in U_{g r t}}\left(\frac{P_{u r t}}{\varphi_{u r t}}\right)^{1-\sigma_{U g}}\right]^{\frac{1}{1-\sigma_{U g}}}. $$

With the price indices for each product group known, we solve for the allocation of expenditure across product groups at a given retail firm. The share of consumer spending on product group g at retail firm r is given by

$$ S_{g r t}{=}\frac{\left(\frac{P_{g r t}}{\varphi_{g r t}}\right)^{1-\sigma_{G}}}{\sum_{k\in G_{r t}}\left(\frac{P_{k r t}}{\varphi_{k r t}}\right)^{1-\sigma_{G}}},\qquad\sigma_{G}{>}0,\qquad\varphi_{k r t}{>}0. $$

The corresponding price index for retail firm r is then given by

$$ P_{r t}=\left[\sum_{g\in G_{r t}}\left(\frac{P_{g r t}}{\varphi_{g r t}}\right)^{1-\sigma_{G}}\right]^{\frac{1}{1-\sigma_{G}}}. $$

Similarly, the share of consumer spending at retail firm r is given by

$$ S_{r t}{=}\frac{\left(\frac{P_{r t}}{\varphi_{r t}}\right)^{1-\sigma_{R}}}{\sum_{s\in R_{t}}\left(\frac{P_{s t}}{\varphi_{s t}}\right)^{1-\sigma_{R}}},\qquad\sigma_{R}>0,\qquad\varphi_{s t}>0. $$

The corresponding price index for the total consumption of the representative consumer at time t is then given by

$$ P_{t}=\left[\sum_{r\in R_{t}}\left(\frac{P_{rt}}{\varphi_{rt}}\right)^{1-\sigma_{R}}\right]^{\frac{1}{1-\sigma_{R}}}. $$

We can now solve for the quantity of each product demanded at each retail firm. The sales of product u at retail firm r are given by

$$ E_{urt}=S_{urt}S_{grt}S_{rt}E_{t}, $$

where $ E_{t} $ is the total expenditure of the representative consumer at time t. The quantity demanded of product u at retail firm r, $ Q_{urt} $ , is, by substituting equation (7) into $ Q_{urt}=E_{urt}/P_{urt} $ and rewriting, given by

$$ Q_{urt}=\varphi_{rt}^{\sigma_{R}-1}\varphi_{grt}^{\sigma_{G}-1}\varphi_{urt}^{\sigma_{Ug}-1}E_{t}P_{t}^{\sigma_{R}-1}P_{rt}^{\sigma_{G}-\sigma_{R}}P_{grt}^{\sigma_{Ug}-\sigma_{G}}P_{urt}^{-\sigma_{Ug}}. $$

#### 3.1.3 Normalization of quality parameters

Service quality perceived by consumers can be divided into the perceived service quality relating to each tier of demand. Product-level service quality is the quality of services related to supplying a specific product. This kind of service quality matters
especially in the case of food and drink. Even if the quality of the product itself is the same across different retail firms, their efforts to sell items with a longer sell-by date may affect their sales. Next, group-level service quality refers to the quality of services that are common to all products within the same product group. Maintaining the appropriate temperature in the cold drinks shelf is an example of such efforts. Finally, firm-level service quality refers to the quality of services that are common to all products and product groups at a retail firm. Accessibility of a firm's stores from consumers' home or from a station, wide parking spaces, hospitality of the sales staff, and sophisticated shop displays are major examples.

In our framework, we expect these different types of service quality to be reflected in the quality parameters for each tier of demand. However, the values of the parameters cannot necessarily be attributed to service quality alone: the product-level parameter $ \varphi_{urt} $ is determined not only by the quality of the service but also by the quality of the product. Similarly, the group-level parameter $ \varphi_{grt} $ is also affected by consumers' relative tastes with regard to different product groups, which are mostly unrelated to retail firms' efforts.

In order to focus on service quality, we assume that the quality of a product itself is the same across retail firms. Clearly, in practice this is not true for some types of goods. The quality of vegetables, fruit, meat, and fish, for example, differs significantly across stores and sometimes even over time in the same store. However, such goods are the exception in our dataset, since it contains only products with a JAN code, the quality of which is highly standardized in most cases. Similarly, we assume that consumers' tastes with respect to the same product or the same product group are the same across retail firms irrespective of retail firms' efforts. Given these assumptions, we expect that differences across retail firms in the quality parameters for the same product or product group are entirely attributable to differences in product-level or group-level service quality.

However, a simple application of the approach employed by Hottman et al. (2016) and Hottman (2016) does not allow us to directly measure these differences. Hottman et al. (2016) and Hottman (2016) normalize the quality parameters by setting their geometric means to one. In the context of our paper, such normalization would be expressed as follows:

$$ \left(\prod_{g\in G_{r t}}\varphi_{g r t}\right)^{\frac{1}{N_{G_{r t}}}}=\left(\prod_{u\in U_{g r t}}\varphi_{u r t}\right)^{\frac{1}{N_{U_{g r t}}}}=1, $$
where $ N_{G_{n}} $ is the number of product groups supplied by retail firm r and $ N_{U_{g_{n}}} $ is the number of products supplied by retail firm r that belong to product group g. Under this normalization, the quality ratio of product u sold at retail firm r and $ \dot{r} $ is given by

$$ \varphi_{u r t}/\varphi_{u\acute{r}t}{=}\frac{P_{u r t}/\tilde{P}_{U_{g r t}}}{P_{u\acute{r}t}/\tilde{P}_{U_{g\acute{r}t}}}\cdot\left(\frac{S_{u r t}/\tilde{S}_{U_{g r t}}}{S_{u\acute{r}t}/\tilde{S}_{U_{g\acute{r}t}}}\right)^{\frac{1}{\sigma_{U g}-1}}, $$

where $ \widetilde{P}_{U_{grt}}=\left(\prod_{v\in U_{grt}}P_{vrt}\right)^{\frac{1}{Nu_{grt}}} $ and $ \widetilde{S}_{U_{grt}}=\left(\prod_{v\in U_{grt}}S_{vrt}\right)^{\frac{1}{Nu_{grt}}} $ . That is, the quality ratio is determined by the ratio of relative prices measured by the average price of the products in group g at each firm (the ratio of $ P_{urt}/\widetilde{P}_{U_{grt}} $ and $ P_{urt}/\widetilde{P}_{U_{grt}} $ ) and the ratio of relative sales measured by the average sales of the products in group g at each firm (the ratio of $ S_{urt}/\widetilde{S}_{U_{grt}} $ and $ S_{urt}/\widetilde{S}_{U_{grt}} $ ). Since the relative prices and relative sales are evaluated in relation to the average prices and sales of all the products in group g sold at each firm, it is not possible to isolate the quality ratio of product u from the prices and sales-and hence indirectly from the product and service quality-of other products.

Therefore, we instead use a benchmark approach in the normalization. We choose a product group, say, group $ \bar{g} $ , that is sold in the broadest range of retail firms as the benchmark group, and a product in each product group, say, product $ \bar{u} $ , that is sold in the broadest range of retail firms as the benchmark product of that group. We assume that the service quality related to product $ \bar{u} $ and group $ \bar{g} $ is the same across the retail firms, namely $ \varphi_{\bar{u}rt} = \varphi_{\bar{u}rt} = 1 $ and $ \varphi_{\bar{g}rt} = \varphi_{\bar{g}rt} = 1 $ for all r and $ \dot{r} $ . The quality parameters of the other groups and products are determined in relation to the quality of the benchmark group and product. Thus, the quality ratio is given by

$$ \varphi_{u r t}/\varphi_{u\acute{r}t}{=}\frac{P_{u r t}/P_{\bar{u}\acute{r}t}}{P_{u\acute{r}t}/P_{\bar{u}\acute{r}t}}\cdot\left(\frac{S_{u r t}/S_{\bar{u}\acute{r}t}}{S_{u\acute{r}t}/S_{\bar{u}\acute{r}t}}\right)^{\frac{1}{\sigma_{U g}-1}}, $$

which is determined by the ratio of relative prices measured by the price of the benchmark product $ \bar{u} $ at each firm (the ratio of $ P_{urt}/P_{urt} $ and $ P_{urt}/P_{urt} $ ) and the ratio of relative sales measured by the sales of $ \bar{u} $ at each firm (the ratio of $ S_{urt}/S_{urt} $ and $ S_{urt}/S_{urt} $ ). Unlike in the approach by Hottman et al. (2016) and Hottman (2016), the
$$ V_{urt}(Q_{urt})=z_{urt}Q_{urt}^{1+\delta_{g}},\quad z_{urt}>0,\quad\delta_{g}>0, $$

where $ Q_{urt} $ is the total quantity supplied of product u by retail firm r; $ \delta_{g} $ determines the convexity of marginal costs with respect to output for products in product group g; and $ z_{urt} $ is a firm-product-specific shifter of the cost function. Costs are incurred in terms of a composite input that is chosen as the numéraire. One reason why we assume that $ \delta_{g}>0 $ is the presence of fixed factors in the retail production function. This type of convex cost function is also generated by inventory-capacity problems (Gallego et al., 2006). The same kind of cost function at the product level is used by Burstein and Hellwig (2007) and Broda and Weinstein (2010). The marginal cost of supplying product u is given by

$$ m_{u r t}=(1+\delta_{g})z_{u r t}Q_{u r t}^{\delta_{g}}. $$

In addition, each retail firm must also pay a fixed market access cost of $ H_{t}>0 $ .

#### 3.2.2 The retail firm’s problem

The total profit of retail firm r at time t is thus given by

$$ \pi_{r t}=\sum_{g\in G_{r t}}\sum_{u\in U_{g r t}}\left[P_{u r t}Q_{u r t}-V_{u r t}(Q_{u r t})\right]-H_{t}. $$

In the case of Bertrand competition, each retail firm chooses its prices $ \{P_{uri}\} $ to maximize profits. The first order conditions take the following form:

$$ Q_{urt}+\sum_{g\in G_{rt}}\sum_{v\in U_{grt}}\Big[P_{vrt}\frac{\partial Q_{vrt}}{\partial P_{urt}}-\frac{\partial V_{vrt}(Q_{vrt})}{\partial Q_{vrt}}\frac{\partial Q_{vrt}}{\partial P_{vrt}}\Big]=0 $$

Solving the first order conditions allowing retail firms to internalize their impact on the price index, the optimal price is given by

$$ P_{urt}=\mu_{rt}m_{urt}. $$

$ \mu_{rt} $ is the markup over marginal cost and is given by

$$ \mu_{r t}=\frac{\varepsilon_{r t}}{\varepsilon_{r t}-1}, $$

where $ \varepsilon_{rt} $ is retail firm r's perceived elasticity of demand,

$$ \varepsilon_{r t}=\sigma_{R}-(\sigma_{R}-1)S_{r t}. $$

In the case of Cournot competition, the perceived elasticity becomes

$$ \varepsilon_{r t}=\frac{1}{1/\sigma_{R}-(1/\sigma_{R}-1)S_{r t}}. $$