This page contains pedagogically-oriented material on maximum entropy and exponential models. The emphasis is towards modelling of discrete-valued stochastic processes which arise in human language applications, such as language modelling.
All links point to postscript files unless otherwise indicated.
This is a high-level tutorial on how to use MaxEnt for modelling discrete stochastic processes. The motivating example is the task of determining the most appropriate translation of a French word in context. Where examples are called for, we draw from applications in human language technology. A gentle introduction to iterative scaling This note concerns the improved iterative scaling algorithm for computing maximum-likelihood estimates of the parameters of exponential models. This tutorial explains how to build maximum entropy models for natural language applications such as information retrieval and speech recognition.