Futures & Market Research Glossary

Percentile rank is a relative-position statistic that expresses where an observation stands within a specified reference distribution, usually on a 0-to-100 scale. A percentile rank near 90 places the observation near the upper tenth of that reference distribution under the chosen ranking convention. It describes order within a comparison set—not how far the observation is above average, and not the probability of any future market outcome.

The convention is part of the number

There is no single finite-sample formula used by every statistical package. One common empirical-distribution convention assigns 100 times the fraction of observations at or below the value. Other implementations can use different endpoint formulas, denominators, interpolation rules or treatments of ties. In a 20-observation window, a simple rank-divided-by-sample-size construction moves in five-percentage-point steps before special tie handling or interpolation. Reproducible research therefore needs to specify the reference sample, window length, ranking direction, missing-value and tie treatment, and the percentile-rank convention.

Percentile rank and percentile ask inverse questions rather than describing the same quantity. Percentile rank starts with a value and asks where it sits in the distribution; a percentile starts with a probability or percentage and asks which value corresponds to that location. In finite samples, software can implement these mappings with related but not identical conventions.

Relative position is not distance

In market research, a rolling percentile rank can place an indicator, spread, volatility measure, valuation metric, or positioning series in historical context. A reading at the 95th percentile rank is extreme relative to the chosen window, but it does not say how far the value is from the center of the distribution. Rank-based normalization preserves ordering under a strictly increasing rescaling, which helps compare differently scaled series, but it discards information about spacing between observations.

That trade-off distinguishes percentile rank from a Z-Score. A Z-Score measures distance from a mean in standard-deviation units; percentile rank measures relative order. The same current value can receive materially different percentile ranks under a 60-day versus 252-day window, or after a structural regime change. A high or low percentile rank should therefore be read as contextual evidence about the specified sample, not as a standalone bullish, bearish, reversal, or probability signal.

Physical market tightness is a condition in which the cushion of readily available, usable supply is limited relative to near-term demand at the place, quality and time buyers need it. It is an inferred market condition rather than a single standardized statistic, and it is not synonymous with a production deficit or a low headline inventory number. Nor is it the same as market balance over a period: a market can appear balanced in aggregate while prompt availability is constrained in a specific location, grade or product.

The relevant stock is the stock that can satisfy prompt demand

The key variable is effective availability. Two markets with the same aggregate inventory can have very different degrees of tightness if one has material near consuming regions, in the required specification, available for sale and accessible through unconstrained logistics, while the other does not. Tightness can therefore be local or quality-specific before it becomes obvious in broad supply statistics. A grain market may have comfortable total stocks but scarce milling-quality wheat; a gas-producing system can be well supplied overall while a consuming region pays a sharp premium because pipeline capacity is constrained.

Prices reveal pressure, but no single indicator proves it

Researchers therefore look for a cluster of evidence rather than one universal measure: inventory draws or low stocks-to-use ratios, prompt physical premiums, widening location or quality differentials, firm nearby calendar spreads or backwardation, and evidence that storage, transport, production or processing buffers are becoming constrained. These observations are related but not interchangeable. Backwardation can be consistent with a high value placed on immediate supply, but futures-curve shape is not direct proof of a physical shortage. Likewise, low inventories need not imply acute tightness if demand is weakening or replenishment is easy.

Physical market tightness is best understood as reduced resilience in the near-term supply system. When buffers are thin, fragmented or hard to mobilize, relatively small disruptions can produce disproportionate moves in basis, location or quality differentials, and prompt prices because additional usable supply may arrive only after a delay. The practical question is therefore not merely “How much inventory exists?” but “How much of the right material can reach the marginal buyer, soon enough, under current operating constraints?”

Producer/Merchant/Processor/User is one of the four reportable trader classifications in the CFTC’s Disaggregated Commitments of Traders Report, which covers physical commodity contracts. It describes an entity that predominantly engages in the production, processing, packing or handling of a physical commodity and uses futures markets to manage or hedge risks associated with those activities. The other Disaggregated categories are Swap Dealers, Managed Money and Other Reportables. The classification is not interchangeable with the broader Legacy COT “Commercial” category: Legacy Commercial combines types of commercial traders that the Disaggregated report classifies separately, including Producer/Merchant/Processor/User and Swap Dealers.

Classification follows the trader, not each position

The central interpretive point is that the CFTC classifies the trader rather than assigning a motive to each individual position. Trader categories are based on predominant business purpose reported on Form 40 and reviewed by CFTC staff, which may reclassify a trader when other information about its market use warrants it. Current CFTC documentation states that a Form 40 classification is specific to a contract market and typically applies to all of that trader’s positions in that market rather than being assigned position by position. A Producer/Merchant/Processor/User can therefore hold positions that are hedging, speculative or part of a broader risk-management book; the public category does not reveal the motive of each position. Classifications can also change when a trader’s activities, markets used or the information available to the CFTC changes.

Long and short do not reveal the whole portfolio structure

The Disaggregated report treats this category differently when it displays spreading. Swap Dealers, Managed Money and Other Reportables have separate long, short and spreading columns. Producer/Merchant/Processor/User open interest is published only as long or short, and the same trader may appear in both columns. A net series derived by subtracting aggregate shorts from aggregate longs can therefore be useful for measuring the category’s directional balance, but it collapses gross and potentially offsetting exposures. The absence of a separate spreading column is a reporting convention; it is not evidence that these firms never hold spread or offsetting positions.

Researchers also need a historical boundary. The Disaggregated historical series was backcast to June 13, 2006, but the CFTC did not maintain a historical record of large-trader classifications; classifications available during the backcast were applied to earlier positions, and the agency notes that this reduces classification accuracy farther back in time. The category is published in both futures-only and futures-and-options-combined formats, so comparisons should keep report format consistent. Used with these limits, Producer/Merchant/Processor/User positioning is best read as aggregate positioning by entities whose predominant business is tied to the physical commodity chain—not as a position-by-position record of hedging intent or a mechanical bullish or bearish signal.

Production is the flow of physical commodity output recorded during a specified period under a defined measurement boundary. Depending on the market, that may mean crops harvested, minerals or energy extracted, or a processed commodity produced. It is a flow, not a stock. In a commodity balance, production is normally only one source of availability alongside beginning inventories and imports, so production and total supply are not interchangeable.

The measurement boundary can change the number

A production series is meaningful only when the stage of the physical chain is clear. USDA NASS, for example, defines crop production as the quantity recovered or removed from the field and calculates net harvested production from harvested acres and net harvested yield. EIA natural-gas statistics distinguish gross withdrawals, marketed production and dry production because reinjection, venting and flaring, nonhydrocarbon gases, processing losses and liquids extraction affect which volume remains in each series.

This matters when comparing markets, regions or history. Unit conventions, moisture or quality standards, geographical scope, primary versus processed output and methodology revisions can alter comparability. A statistical jump caused by a changed reporting boundary is not the same economic event as more usable commodity reaching the relevant market.

Production enters the balance, not the price in isolation

Production affects market tightness through its interaction with consumption, exports, imports and inventories. More output can loosen a balance if other components are unchanged, but the price effect depends on inventory buffers, logistics and processing constraints, seasonality, demand conditions and expectations. A rise in production therefore does not by itself imply lower prices, just as a decline does not by itself imply higher prices.

For research, the useful question is not simply whether production rose or fell. It is what quantity the series counts, where in the physical chain it is measured, when it enters the relevant balance, and whether a change reflects actual output, methodology or both.

Publication lag is the interval between the date or period that a dataset describes and the time when that information becomes publicly available. In official statistics this is commonly treated as a dimension of timeliness: a monthly observation may describe activity during one month but not be disseminated until days or weeks after that month has ended. The important distinction is that an observation’s reference date is not necessarily its information-availability date.

The timestamp that matters depends on the question

For descriptive analysis, organizing data by reference period may be entirely appropriate. For historical trading research, event studies or signal construction, however, the relevant timestamp is usually the first time the information was publicly available. Suppose a value describes March but is first released in April. A backtest that allows the March value to influence a March decision has used information that a real participant could not yet have observed. Publication lag is therefore not itself look-ahead bias, but ignoring it can create look-ahead bias.

The lag is not necessarily a fixed number of calendar days. It can vary with the dataset, reporting frequency, compilation process, business-day calendar and release schedule. Researchers should therefore prefer actual historical release metadata where available rather than mechanically shifting every observation by an assumed constant delay. When a release has a specified publication time, date-only alignment can also be too coarse for intraday or same-day analysis.

Publication lag is different from revision

Publication lag concerns when an observation first becomes available; data revision concerns how a value changes after it has already been published. Real-time datasets can preserve both dimensions by recording successive vintages: the same reference period may first appear on one date and then receive different values on later dates. A database containing today’s best historical estimates therefore does not necessarily reproduce the information set that existed at an earlier decision date.

There is also a practical distinction between official publication and downstream data delivery. A statistical agency may release a figure at one time while a particular database or vendor makes it available later. For research concerned only with public information, the official release timestamp may be the relevant boundary; for a simulation of a specific executable data pipeline, the later usable timestamp may matter as well.

For point-in-time research, the safest question is not simply “What value belongs to this date?” but “What value had actually been released and was available by this date?” Preserving that distinction is essential when comparing indicators with different release delays, reconstructing historical decisions or testing whether an apparent predictive relationship could genuinely have been observed in real time.