美联储-制造业情绪：用文本分析预测工业生产（英）-2024.4-46页

免费

3.0 babyfacer 2024-05-15 97 16 738.72KB 45 页免费

侵权投诉

Finance and Economics Discussion Series

Federal Reserve Board, Washington, D.C.

ISSN 1936-2854 (Print)

ISSN 2767-3898 (Online)

Manufacturing Sentiment: Forecasting Industrial Production with

Text Analysis

Tomaz Cajner, Leland D. Crane, Christopher Kurz, Norman Morin, Paul E.

Soto, Betsy Vrankovich

2024-026

Please cite this paper as:

Cajner, Tomaz, Leland D. Crane, Christopher Kurz, Norman Morin, Paul E. Soto, and Betsy

Vrankovich (2024). “Manufacturing Sentiment: Forecasting Industrial Production with Text

Analysis,” Finance and Economics Discussion Series 2024-026. Washington: Board of Gov-

ernors of the Federal Reserve System, https://doi.org/10.17016/FEDS.2024.026.

NOTE: Staﬀ working papers in the Finance and Economics Discussion Series (FEDS) are preliminary

materials circulated to stimulate discussion and critical comment. The analysis and conclusions set forth

are those of the authors and do not indicate concurrence by other members of the research staﬀ or the

Board of Governors. References in publications to the Finance and Economics Discussion Series (other than

acknowledgement) should be cleared with the author(s) to protect the tentative character of these papers.

Manufacturing Sentiment:

Forecasting Industrial Production with Text Analysis∗

Tomaz Cajner

Norman Morin

Leland D. Crane

Paul E. Soto

Christopher Kurz

Betsy Vrankovich

April 2024

Abstract

This paper examines the link between industrial production and the sentiment ex-

pressed in natural language survey responses from U.S. manufacturing ﬁrms. We com-

pare several natural language processing (NLP) techniques for classifying sentiment,

ranging from dictionary-based methods to modern deep learning methods. Using a

manually labeled sample as ground truth, we ﬁnd that deep learning models—partially

trained on a human-labeled sample of our data—outperform other methods for clas-

sifying the sentiment of survey responses. Further, we capitalize on the panel nature

of the data to train models which predict ﬁrm-level production using lagged ﬁrm-level

text. This allows us to leverage a large sample of “naturally occurring” labels with no

manual input. We then assess the extent to which each sentiment measure, aggregated

to monthly time series, can serve as a useful statistical indicator and forecast industrial

production. Our results suggest that the text responses provide information beyond

the available numerical data from the same survey and improve out-of-sample forecast-

ing; deep learning methods and the use of naturally occurring labels seem especially

useful for forecasting. We also explore what drives the predictions made by the deep

learning models, and ﬁnd that a relatively small number of words—associated with

very positive/negative sentiment—account for much of the variation in the aggregate

sentiment index.

JEL codes: C1, E17, O14

Keywords: Industrial Production, Natural Language Processing, Machine Learning,

Forecasting

∗All authors are at the Federal Reserve Board of Governors. We thank the Institute for Supply Manage-

ment, including Kristina Cahill, Tom Derry, Debbie Fogel-Monnissen, Rose Marie Goupil, Paul Lee, Susan

Marty, and Denis Wolowiecki, for access to and help with the manufacturing survey data that underlie the

work described by this paper. We are thankful for comments and suggestions from Stephen Hansen, Andreas

Joseph, Juri Marcucci, Arthur Turrell, and participants at the Society for Government Economists Annual

Conference, the ESCoE Conference on Economic Measurement, the Government Advances in Statistical

Programming Conference, the Society for Economic Measurement Conference, and the Nontraditional Data,

Machine Learning, and Natural Language Processing in Macroeconomics Conference. The analysis and con-

clusions set forth here are those of the authors and do not indicate concurrence by other members of the

research staﬀ or the Board of Governors.

1 Introduction

In recent years there has been an explosion of interest in natural language processing (NLP)

within ﬁnance and macroeconomics. The use of text data to forecast and assist in model

estimation is becoming increasingly commonplace. Still, there are many open questions

around the use of NLP in empirical work. For example, which of the numerous available

methods work best, and work best in speciﬁc contexts? Are oﬀ-the-shelf tools appropriate,

or are there greater returns to specializing models to the data at hand? How useful is

text for forecasting real output indicators, such as manufacturing output? What explains

the predictions made by complicated NLP models? This paper addresses these questions,

using a novel dataset and a variety of NLP methods ranging from traditional dictionaries to

ﬁne-tuned transformer neural networks.

Our primary data source is the monthly survey microdata underlying the Institute for

Supply Management’s (ISM) Manufacturing Report on Business. The survey is taken by

purchasing managers at a representative sample of U.S. manufacturing ﬁrms. Part of the

survey consists of categorical-response questions about aspects of their current operations,

including production, inventories, backlogs, employment, and new orders. The answers to

these questions are of the form “worse/the same/better than last month”, and are aggregated

into the widely-reported ISM diﬀusion indexes. But the survey also includes free-response

text boxes, where purchasing managers can provide further comments either in general or

about speciﬁc aspects of their businesses; these comments are a novel source of signal about

the economy and our focus in this paper.1

Our ﬁrst step is to quantify the text into an economically important and interpretable

measure. We focus on sentiment, given that waves of optimism and pessimism have his-

torically been linked to business cycle ﬂuctuations (Keynes, 1937). We begin by evaluating

various NLP methods in terms of their ability to correctly classify the sentiment expressed

in individual comments. Our context is fairly speciﬁc: the data are manufacturing-sector

purchasing managers opining about about the business outlook for their ﬁrm, without much

discussion of ﬁnancial conditions. While there are numerous sentiment classiﬁcation mod-

els available, many were developed with other data in mind, such as social media posts

(Nielsen, 2011). Even within economics and ﬁnance, most work has focused on ﬁnance-

1While ISM collects these responses through the survey, this text is conﬁdential and not incor-

porated into the publicized indexes. A sample of responses are published in the monthly ISM Re-

port on Business (see https://www.ismworld.org/supply-management-news-and-reports/reports/

ism-report-on-business/).

研报加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载研报到电脑，查找使用更方便

免费 16人已下载

立即下载

标签： #美联储

摘要：

FinanceandEconomicsDiscussionSeriesFederalReserveBoard,Washington,D.C.ISSN1936-2854(Print)ISSN2767-3898(Online)ManufacturingSentiment:ForecastingIndustrialProductionwithTextAnalysisTomazCajner,LelandD.Crane,ChristopherKurz,NormanMorin,PaulE.Soto,BetsyVrankovich2024-026Pleasecitethispaperas:Cajner,To...

展开>> 收起<<

美联储-制造业情绪：用文本分析预测工业生产（英）-2024.4-46页.pdf

共45页,预览10页

还剩页未读，继续阅读