TAG Unit 3.10.4: Variable Demand Modelling - Convergence Realism and Sensitivity

June 2006

1. Variable Demand Modelling - Convergence Realism and Sensitivity

1.1 Background
1.2 Use of DIADEM
1.3 DIADEM Procedures and Convergence
1.4 Other Software
1.5 Realism Testing
1.6 Sensitivity Testing
1.7 Main Changes from Existing Advice

2. Further Information

3. References

4. Document Provenance

1. Variable Demand Modelling - Convergence Realism and Sensitivity

1.1 Background

1.1.1 TAG units 2.9 and 3.10 explain why variable demand modelling needs to be considered and provide guidance on how to carry out such modelling for highway schemes. This unit forms stage 4 of the overall process; TAG units 3.10.1, 3.10.2 and 3.10.3, detail the previous stages.

1.1.2 Important recommendations are shown highlighted and in bold. If those actions are not followed, analysts will need to provide rigorous justification for the course of action taken.

1.2 Use of DIADEM

1.2.1 Once the extent of the necessary variable demand modelling has been decided, the next question is how to build the model. Few users will have the resources available to build their own model from scratch, nor is this a sensible approach unless there is a need for a general-purpose model for policy analysis much wider than assessment of an individual scheme. Most users will need to identify available software packages which incorporate the necessary demand mechanisms, and can accept the data and parameters appropriate to the scheme under study. In most cases these mechanisms are provided by way of macros developed for each study, rather than as generic model formulations.

1.2.2 In tandem with the development of this TAG unit, software called DIADEM: Dynamic Integrated Assignment and Demand Modelling has been developed to provide simple multi-stage demand models and an interface with commercially available packages.

1.2.3 The DIADEM procedures provide an adjustable hierarchical structure of trip frequency, mode choice, distribution, and time of day choice, and an interface to assignment. It is also possible to use DIADEM for a simple "own-cost" elasticity calculation where applicable. The DIADEM framework controls iteration within assignment, and between demand and assignment, to ensure that the calculation reaches an acceptable equilibrium.

1.2.4 At present, DIADEM has been developed with an interface to the CONTRAM and SATURN assignment packages. It is expected, however, that the suppliers of other assignment packages will provide equivalent functionality or suitable interfaces with DIADEM, so that it can be used with whatever assignment model is available for the scheme to be assessed. As ever, there is no monopoly on the most convenient way to achieve best practice: as with choice of assignment package, which one to adopt is a matter of individual preferences and priorities. If a decision is made to use DIADEM then Section 1.3 provides a summary of the approach. If other software is to be used then Section 1.4 gives guidance on how alternative software should be used.

1.3 DIADEM Procedures and Convergence

1.3.1 The DIADEM procedures cover all the demand-side issues that must be considered when applying multi-stage models, as described in VDM Key Processes (Unit 3.10.3), and provides the user with the necessary choice between alternative formulations and full control over each aspect.

1.3.2 Model Type: the model can be incremental at present although an absolute alternative is ultimately envisaged to every response in the overall application.

1.3.3 Demand responses: DIADEM allows the following responses to be included:

• elasticity model: the elasticity model has a 2-parameter Tanner form, which is intended to be used in its extremes, setting one of the parameters to 0 to return either a power or an exponential form, as discussed in VDM Key Processes (Unit 3.10.3). The use of both parameters together in the Tanner form is not recommended.
• trip frequency model: the trip frequency model is an exponential elasticity function, but unlike general elasticity models (which operate at the OD level), the trip frequency model applies to the zone level, using composite zone (accessibility) costs as discussed in VDM Key Processes (Unit 3.10.3).
• modal choice model: the mode choice model is a binomial logit formulation. The model does not automate the hierarchical modelling of public transport modes i.e. sub mode choice. Any lower-level split between different public transport modes would have to be estimated independently or at assignment as discussed in VDM Key Processes (Unit 3.10.3). However, this sub-mode choice will rarely be needed in a road-based scheme appraisal.
• distribution model: the trip distribution model can be singly-origin constrained or singly-destination constrained (to approximate to Production /Attraction constraints) or doubly-constrained as discussed in VDM Key Processes (Unit 3.10.3).
• time of day choice model: a macro time period choice model in logit form is currently available. Micro time period choice using HADES is planned for future release.

1.3.4 There are still a number of technical issues to be resolved, concerned with incremental formulation of the HADES model, the composite cost formulae, and the use of scheduling costs. For a discussion on alternative model forms of departure-time choice, see VDM Key Processes (Unit 3.10.3) and Batley et al (2001).

1.3.5 Model hierarchy: DIADEM allows a different range of responses, a different model form and a different hierarchy to be applied to each individual purpose and traveller type combination.

1.3.6 Model parameters: Each modelled response is driven by a single user-defined ? parameter, per purpose and traveller type combination, apart from the elasticity models which as Tanner formulations are driven by 2 parameters (though normally one of these should be set to zero, returning the Tanner function to either a power or an exponential function). Calibration areas with potentially different parameters in each area will ultimately be provided.

1.3.7 Generalised costs: Generalised cost coefficients are defined for each purpose and traveller type combination, allowing for time, distance and monetary components. Any weighting of in-vehicle, waiting or walking time must be done within the assignment stage or outside the DIADEM environment, and a similar argument applies to crowding effects. The analyst will be provided with a flexible tool in DIADEM to arrange the sequence of demand responses and warned if the proposed sequence does not reflect demand sensitivities. However, the analyst will still need to ensure that the definitions of generalised cost in the demand and assignment phases include all the necessary terms and are sufficiently compatible.

1.3.8 Model running: Guidance on how to run the DIADEM software is given in the user guide to the software. It should be noted that the solution closest to equilibrium may not necessarily be the one produced by the last iteration of the DIADEM demand/assignment modelling system. The solution from the iteration with the lowest GAP value should be used for appraisal purposes. This may require an additional run of the assignment package using this 'best' trip matrix to obtain a converged solution.

1.4 Other Software

1.4.1 Section 1.3 outlined the use of the DIADEM procedures. However, using DIADEM is not the only approach that could be adopted and the practitioner may wish to develop an approach based on existing software packages that have not developed interfaces with DIADEM. Such software is being developed continually, and only the more widely used packages are mentioned here.

1.4.2 There are a number of different approaches than can be adopted when using non DIADEM procedures:

1.4.3 Combined assignment-demand models. Firstly, one can use other software that can handle demand-supply responses as a combined assignment-demand model. This is the preferred alternative solution to using DIADEM in most cases since the software will be constructed to ensure, as far as possible, that the model is properly integrated, computationally efficient and sufficiently converged to a correct solution. At present the main drawback is that the demand responses that can be modelled may be limited, either in number or in the sequence order. For example, in the case of the SATURN software package, assignment can be combined with elastic demand and one or both of mode-choice and singly constrained trip distribution. The approach, at present, does have the disadvantage that the sequencing allowed is restricted. Similarly, the EMME/2 transportation suite allows a combined model of trip assignment with elastic demand or mode-choice through the use of built-in modules but other combinations need to use individual models of demand and assignment constructed by the user using the suites macro language. With these models, more exact measures of convergence can be defined than the % relative GAP measure recommended in Section 1.5. This arises from the nature of the combined demand-supply formulation (such as that used by the SATURN program). At present, no fixed level to be obtained from such measures has been set but details of the approach to improve convergence should be given in the Validation Report. (The latest version of the SATURN suite can give values from which to calculate the % GAP function.). The TUBA tests outlined in the next section can be done to ensure that convergence is to the level required by the scale of the scheme.

1.4.4 Combining separate demand and assignment models: The next best alternative is to make use of a transportation suite software's matrix manipulation software to construct the model of demand responses oneself and iterate between the assignment program and the demand model. This will at least ensure that the supply and demand data are in compatible format. However, it is likely to require expertise both in transportation modelling and in knowledge of the workings of the transportation software. Various transportation software packages are available which cover both demand and assignment functions. The most widely-used in the UK are probably the Citilabs' TRIPS transportation suite (now part of the CUBE family of models), which includes public transport and highway assignment and demand functions within the same modelling environment, the EMME/2 suite as noted above and PTV's VISEM (in the form of tours instead of trips) and VISUM suites will also perform similar functions.

1.4.5 The user in these circumstances will need to ensure that that the final solution meets the convergence criteria set out in Section 1.5. To do this, it may necessary to devise a sophisticated approach to cycling between the assignment and demand responses that will ensure a stable converged solution in a reasonable time. The most common solutions are simply to iterate between the converged assignment model and the demand response model, passing travel costs from the assignment model to the demand model and passing trips (or most commonly vehicle trips) from the demand model to the assignment model. However, this approach is not guaranteed to converge, except when using techniques such as the Method of Successive Averages (MSA). Both techniques can, in some circumstances, take a long time to reach a sufficiently converged solution. (This is one of the reasons that algorithms that incorporate both the demand and assignment phases in one procedure have been developed.) It is difficult to provide a single ready-made solution for all software and all schemes, and it may be necessary to seek advice from the software developers. For these models the iteration between supply (assignment) model and demand models should give statistics from which to calculate the % GAP statistic, as recommended in section 1.5. The approach to iterating between the supply and demand models, and the monitoring of the convergence progress should be detailed in the Validation Report. Evidence should be provided that the models meet the convergence requirements set out in Section 1.5 (1.5.7).

1.4.6 Combining models from different transportation suites: In some cases the assignment package and the software used to model the demand responses may not be from the same package, or software used in highway assignment may be different from that used for public transport assignment. Examples of this can be found in many of the Multi-Modal Studies, for example using SATURN for the highway assignment and TRIPS for the demand. This should not, in itself, prove too much of a problem provided that care is taken in transferring data from one package to another. In other Multi-Modal Studies the demand modelling was split between two different software packages, for general road traffic and public transport. However, the difficulties in transferring data from one transportation suite to another should not be underestimated and time and resources should be allowed to ensure that the data is compatible.

1.4.7 Users are responsible with most software for arranging the sequence of demand responses and for ensuring that the proposed sequence reflects demand model sensitivities. Analysts also need to ensure that the definitions of generalised cost in the demand and assignment phases include all the necessary terms and are sufficiently compatible. They also need to ensure that a stable, converged, solution to both the assignment and demand responses is produced in a reasonable time. This is particularly important when assessing small schemes with large models. For these models the iteration between supply (assignment) model and demand models should give statistics from which to calculate the recommended convergence statistic. The approach to iterating between the supply and demand models and the monitoring of the convergence progress should be detailed in the Validation Report.

1.4.8 Whatever approach is adopted the user should ensure that documentation is provided in sufficient detail for a third party to follow all the steps and the convergence properties of the combined model are detailed.

1.4.9 Convergence: This is a key to achieving good modelling practice. High levels of convergence should be achieved in any assignment modelling. This applies to both small networks and, especially, large networks where small percentage changes in convergence may result in large changes in flows and times around a potential scheme. In addition, the demand - supply convergence should be monitored by using the convergence measure (% relative GAP) defined in 1.5.2, with the aim of reaching the convergence level determined by para 1.5.7. If that cannot be reached then a convergence level of at least 0.2% is recommended.

1.4.10 In summary, if an integrated approach similar to DIADEM is not used the main issues are:

• That the approach adequately addresses the issues of correct sequencing of demand responses and consistency in the definitions of generalised cost in the demand and assignment phases.
• Where a combined assignment/demand model is used, that the convergence statistics output at the end of the modelling meet the requirements set out in Section 1.5 below.
• Where a combined model is not used or not applicable, then care needs to be taken over the cycling between assignment and demand modelling modules so that convergence in travel costs and trips is reached in a reasonable time.
• Sufficient documentation is provided that the approach adopted can be understood and the convergence properties of the model clearly stated.

1.5 Convergence

1.5.1 The original impetus for DIADEM was in the need to improve convergence of demand-supply models, and DIADEM procedures have internal capabilities to apply a range of convergence improving techniques, guided by a number of convergence measures and desired stop criteria. Preliminary tests indicate that improved demand convergence can reduce the convergence errors to less than 10% of the economic benefit. Demand modelling software, such as DIADEM may provide a number of measures of convergence, both relating to proximity (how close to the true equilibrium), and stability (how much the results are changing each iteration). For our purposes the proximity measures are the more important.

1.5.2 The recommended criterion for measuring convergence between demand and supply models is the demand/supply gap defined by:

Where;

X_ijctm is the current flow vector or matrix from the model

C(X_ijctm) is the generalised cost vector or matrix obtained by assigning that matrix

D(C(X_ijctm)) is the flow vector or matrix output by the demand model, using the costs C(X_ijctm) as input

_ijctm represents origin i, destination j, demand segment/user class c, time period t and mode m

Measuring convergence between demand and supply

This is a measure of how far the current flow is from the equilibrium point and will be zero in a perfectly converged model. The demand-supply gap is represented, for one flow, by the shaded area in the figure above. As convergence improves, and the difference in trips between successive iterations decreases so the shaded area decreases until the equilibrium point is reached. One of the reasons for the choice of this statistic is that it is easily calculated and is not dependant on the precise form of demand-supply modelling undertaken. It is referred to as the %GAP, reflecting its relative nature.

1.5.3 The demand/supply model used may report other measures of convergence. Some of these may be stability statistics that indicate how much the solution is changing from one iteration to the next. An example of this could be the maximum change in flows. It is often assumed that a stable solution implies convergence. However, it can also be an artefact of the particular algorithm being used so stability statistics are, in general, not a good indicator of how close the solution is to equilibrium.

1.5.4 The demand/supply gap, as defined in 1.5.2 above, is the most appropriate measure for gauging the error in economic benefit calculations caused by imperfect convergence. It can be calculated for any variable demand/supply modelling system and is not dependant on the form of demand/supply modelling approach.

1.5.5 Tests indicate that gap values of less than 0.1% can be achieved in many cases, although in more problematic systems this may be nearer to 0.2%. Where the convergence level, as measured by the %GAP, is over 0.2% remedial steps should be taken to improve the convergence, by increasing the assignment accuracy.

1.5.6 Convergence and scheme benefits. The required level of convergence needs to be linked to the scale of the benefits of the scheme being appraised, relative to the network size. For instance the calculation of benefits from small schemes in large networks will be much more sensitive to convergence than large schemes in small networks. On the basis of testing it has been discovered that ideally the user benefits, as a percentage of network costs, should be at least 10 times the % Gap achieved in the Do-Minimum and Do-Something scenarios. The estimation of user benefits can be estimated either by using matrix manipulation of the with and without scheme trip and skimmed generalised cost matrices to produce an estimate of the consumer surplus by the rule of a half, or by using the DfT's TUBA program. In either case the worst case convergence of the with and without scheme runs should be taken as the one to compare with the size of the benefits.

Example:

1.6 Realism Testing

1.6.1 Once a variable demand model has been constructed, it is essential to ensure that it behaves "realistically", by changing the various components of travel costs and times and checking that the overall response of demand accords with general experience. If it does not, then the values of the parameters controlling the response of demand to costs should be adjusted until an acceptable response is achieved. This recognises the large and unavoidable uncertainties in some of the parameter values, and the importance of reflecting local conditions in relative values.

1.6.2 By the time this Advice has been digested and the model constructed, it will be apparent that many of the parameters controlling the behaviour of the model have to reflect local circumstances. However, even if there is adequate local data to achieve a good calibration of the model, a fitting of the model to the travel patterns between various pairs of zones (usually referred to as seen in cross-section) does not necessarily guarantee that it will be a good predictor of the response of demand to changes in travel costs over time.

1.6.3 In other cases a local calibration may not have been possible, and it may have been necessary to resort to importing values for the parameters or using "illustrative" values, in which case it is obviously important to check that the behaviour these give rise to is plausible in the new context. However before any parameters in the variable demand model are adjusted the base model (and trips matrices) should have been fully validated and the results outlined in a validation report (see section 1.8).

1.6.4 If the model does not behave in accordance with past experience, it should not be used to appraise a transport scheme, unless a convincing case can be made to explain the differences in terms of special local circumstances. Instead, the model parameters and calibration areas should be modified until its responses are plausible (see 1.6.15 below). The outputs of the realism tests should be disaggregated to see if particular elements of the model are causing problems. If so then the lambda parameters should be changed in such a manner as to produce plausible results, provided that the hierarchy rules set out in VDM Key Processes (Unit 3.10.3) are followed. Such changes should be noted in the Validation Report.

1.6.5 Demand elasticites: The acceptability of the model's responses is determined by the demand elasticities it predicts. These are measured by changing a cost or time component by a small proportionate amount and calculating the proportionate change in trips made. The elasticity recommended which is the arc elasticity formulation is:

e = (log(T¹)-log(T⁰))/(log( C¹)-log(C⁰))

where the superscripts 0 and 1 indicate values before and after the change in cost respectively. For example, if car fuel costs increased by 10% and trips by car fell by 2%, then the elasticity of car trips with respect to fuel price would be -0.2. The same calculation can be made with respect to car-kms travelled rather than to trips, to obtain a distance elasticity.

1.6.6 Components to be tested: Any component of cost or travel time can be checked in this way, but it should be realised that they are not all independent, so that there may be little point in checking all separately. The different components of generalised cost for any particular journey are interlinked by the weights applied in calculating the generalised cost (see VDM Key Processes (Unit 3.10.3)). Thus if one weighted component accounts for twice as much as another in the total cost, the elasticity of demand relative to it will always be twice as much. Nevertheless, it is desirable to test the more important components in this way to ensure that the formulation of generalised cost in the model is correct.

1.6.7 The tests discussed here refer to the demand model: the responses will be modified by iteration with the assignment model, especially where congestion limits traffic growth on some parts of the network, but for realism testing it may be less necessary to establish a close convergence.

1.6.8 As a minimum, analysts should check the elasticity of demand with respect to:

• car fuel cost
• car journey time (this is linked to the journey-time elasticity via the car operating costs formula as a function of distance and speed, and by the value of time, but it is sensible to test both time and cost components separately)
• if modal split to public transport is a significant factor in the modelling, check the elasticity of public transport demand against public transport fares
• if parking is a significant factor, check the response of demand to parking charges.

1.6.9 The check should examine the response of demand for each category of travel, and in each time period. Obviously, there is overlap between these two aspects, but the check will ensure that there has been no misallocation between the categories.

1.6.10 Realism testing involves changing the important aspects that affect demand, whether globally or for selected trips, and ensuring that the response seems "reasonable" - a subjective judgement, but one which can often pinpoint aspects of the modelling which need attention. For each component to be tested, increase its value by, say, 10% and rerunning the model to obtain the new estimate of demand, either as car-kms or public transport trips.

1.6.11 A number of studies in this country using time-series data on car travel, and fuel prices and costs have shown an elasticity of car use with respect to fuel cost of about -0.3 (see Bradburn and Hyman (2002), Graham and Glaister (2002), Hanly Dargay and Goodwin (2002)) and this value equates well with a review of European research on this topic (TRACE, 1999). A realistic model does not necessarily provide precisely this value of -0.3, which is based upon a national mix of trip purposes and time periods. Variation by journey purpose will show elasticities in the range of -0.1 to -0.4 with employer's business trips having values close to -0.1 and the more discretionary trip purposes nearer -0.4.

1.6.12 For most models used in his country, journey time elasticities can be related to the definition of generalised costs but the output elasticities will vary much more than the fuel cost elasticities. The output elasticities should be checked to ensure that the model does not produce very high output elasticities (say greater than -2.0).

1.6.13 Elasticities of public transport trips with respect to public transport fares have been found to lie typically in the range -0.2 to -0.4 for changes taking place within 12 months, and up to -0.9 for changes over a longer period. (TRL, 2004) with those in the peak, or for more obligatory purposes, at the lower end and those in the off-peak, or for more discretionary purposes, at the higher end. These values apply to the totality of public transport passengers; arguably, those with a car available would be expected to show a greater elasticity since they have greater choice, but there is little consistent evidence on what values are appropriate.

1.6.14 Elasticities relative to parking charges will depend upon what fraction of the total generalised costs parking costs account for, and this will depend upon the fraction of motorists who pay for parking as well as on the general level of charges. Judgement of the plausibility of the model response will be very subjective, but unreasonable model response is generally readily identified.

1.6.15 If the "realism" tests suggest elasticities or model responses which are unconvincing, it will be important to experiment with adjusting the model to obtain more appropriate responses. There is a temptation to explain away unusual modelling results in terms of special circumstances applying to the particular modelling context, but unless such an explanation is very convincing it is better to adjust the model's responses to values which are more generally accepted, especially in circumstances where the parameters may have been imported from other applications. The adjustment involves changing the values of the lambda parameters to weaken or strengthen the response, as required - taking care to adjust the hierarchy as necessary.

1.6.16 If the unacceptable responses refer only to particular categories of travel then only the lambdas for these categories need be adjusted. In general, it will not be clear which stages of the multi-stage variable demand calculation should be changed, though if the model contains a time of day choice stage and the unacceptable behaviour lies in the differences between peak and off-peak then attention will obviously be paid to this mechanism. Otherwise, it will be sensible to adjust both distribution and mode choice parameters in the same proportion, unless sensitivity testing (see Section 1.7) suggests that one of these is unreasonably sensitive. Since elasticities scale with the lambda values, if the model elasticity is too large then reduce the lambda values, and vice versa.

1.7 Sensitivity Testing

1.7.1 Sensitivity testing, as distinct from realism testing, is aimed at identifying the relative effects of the various parameters on the outcome of a scheme appraisal, rather than in checking the model responses against experience. Especially where the model parameter values are uncertain it is important to know how sensitive the appraisal results are to these uncertainties, so that confidence can be invested in the conclusions.

1.7.2 The realism testing of Section 1.6 was aimed at ensuring that the model's responses were consistent with previous experience of travel demand and the way it responds to changes in travel costs. Even if the model is satisfactory in these respects, there may still be considerable uncertainty attached to some of its forecasts because of uncertainty in its parameter values. It is important to quantify the effects of this on scheme appraisal, as far as possible, so that the final conclusions on a scheme's value can be robust against these uncertainties. This can be investigated by sensitivity testing of the model's behaviour against variation in those parameters which are judged to:

• have a substantial effect on the model's prediction of changes, and
• be uncertain in their calibration.

1.7.3 The most obvious values are the sensitivity parameters that govern the individual demand mechanisms (i.e. the lambda values). If they have been calibrated on local data, the extent of possible error in their calibration should be examined from the statistics calculated during the fitting, which is usually substantial. If they have been imported, the uncertainty will be even greater since they are being used in a context different from their original application. The illustrative values given in VDM Key Processes (Unit 3.10.3) were obtained from a review of current models, and typically the range of values was twice the mean value. This indicates the degree of uncertainty in values imported from other studies.

1.7.4 If the lambda values have been calibrated on local data, whether for the variable demand model itself or for an existing local model, then check the overall result of the scheme appraisal against runs of the model with the lambdas set at plus the standard deviation of the mean value, or at least +25% of the mean if the actual standard deviation is smaller. Behaviour of the model will not necessarily be symmetrical against increases and decreases in the parameter, but the increase will indicate the strength of the response, and if it is an important factor the result can also be tested against a decrease. If the values have been imported then test the result against +50% of the mean. This range is to reflect the greater uncertainty that occurs with imported values. Unless there are convincing reasons for not doing so, the changes are to be made to all parameters in the same direction at the same time so that the gradation of parameter values is still consistent with the hierarchy.

1.7.5 Given the acknowledged uncertainty of distribution parameters obtained from cross-sectional fitting, this larger margin could be applied even to locally calibrated distribution lambdas. It is the stronger variable demand mechanisms which will have most effect on the assessment, so there may be no point in testing the result against a small trip frequency response, for example, when distribution or mode choice are dominant. Generally, in a scheme aimed at congestion relief, the net benefit will be reduced by increases in the demand for car travel, so that it is the increased lambda values that will test the robustness of the result. If the scheme remains well justified against these higher values then a conclusion that the scheme is beneficial will be robust against the effects of induced traffic. Where the model includes time of day choice it will be essential to test variation of the assumed sensitivity. Evidence for these values is more uncertain and wide sensitivity factors say: +50% and -50% are suggested. The range will be limited by the need to ensure that any changes in the values are still consistent with the hierarchy.

1.7.6 Sensitivity testing should not be limited to the response parameters, however. Any parameter that seems likely to have a substantial effect on the net benefit, and where appreciable uncertainty is likely to affect the assessment substantially, should also be tested. An example of this may be the assumed distribution of willingness-to-pay bands in road-tolling exercises.

1.7.7 There are other sensitivity tests that should be undertaken for forecast years to test the sensitivity of the appraisal to variations in other inputs such as changes in the build-up of demand, values of time, or differing economic forecasts. These tests are described in more detail in the advice on Major Scheme Appraisal in Local transport plans: part 3. Annex F (DfT, 2003).

1.7.8 Although sensitivity testing is important, there is a danger in using it to obtain such a wide range of values that any prediction is mistrusted. In interpreting the results it is important to understand (and to emphasise in presentation) that the central values are still the best available prediction of the likely outcome, and additional forecasts obtained by sensitivity testing are purely to establish the effects of uncertainty around this central forecast. The aim is for the modeller to make clear the extent of the possible uncertainty, while providing clear central predictions to support policy making and assessment.

1.8 Reporting

1.8.1 The results of the realism tests, along with the sensitivity tests discussed in Section 1.7 should be documented in a validation report, either as an addition to the Validation Report required of all road scheme appraisals with regards to model calibration and fit, or as a separate document validating the variable demand aspects of the model. The items that should be included in a Validation Report are set out in significant detail in DMRB Volume 12 Section 2 Part 1.

1.8.2 The items in the Validation Report should include:

• a description of the model used and its development (including evidence of the fit achieved to the calibration data, and a description of any sensitivity tests undertaken, and their results);
• a description of the data used in building and validating the model;
• evidence of the validity of the network employed;
• a validation of the trip matrices employed;
• a validation of the trip assignment;
• a validation of any other special features (e.g. higher tier model inputs, trip end models, modal choice models, etc) employed; and
• a present year validation, if appropriate.

1.8.3 The validation of special features including details of the variable demand model chosen and should include at least the following items:

• The background to the decision on the particular demand responses included in the model. This will include a statement on any demand tests.
• A description of the reasoning behind the choice of lambda parameter values, including any local calibration should be given. The parameter values should be explicitly shown together with details of the elements of generalised cost, and the route-choice factors.
• Where public transport schemes are being considered then the public transport assignment model will need to be validated.
• Details should be given of any realism tests, which should, at least, include the estimation of the elasticity of car travel (trips or kilometres) to changes in car fuel cost and, if possible, to car journey time. Where a mode-choice has been included the realism checks should also include the sensitivity to changing bus/rail fares. The Report should also include details of any changes to the model parameters arising from these tests.

1.8.4 Details should be given of any base year sensitivity tests undertaken.

1.9 Main Changes from Existing Advice

1.9.1 1.1.1 This Section describes those aspects of the Advice where the Department for Transport's expectations of good practice have changed and where departures from existing guidance have been recommended .

1.9.2 The intention of this Advice is to describe the basis of variable demand modelling as clearly and simply as possible, and to recommend simple versions of best current practice. It suggests a minimum set of requirements for testing transport scheme appraisal against the likely response of demand, and is intended to represent generally-accepted practice at this relatively basic level. Consequently, it is not aimed at suggesting fundamental changes to existing practice in demand modelling. However, because it is intended for application in the wider area of scheme assessment where, until recently, the response of travel demand to a scheme was often considered rather cursorily, if at all, it does represent a significant step forward in general appraisal practice, and a change in the Department for Transport's expectations of good practice.

1.9.3 The main points to note are the following:

• Overall, there should now be a presumption that the effects of variable demand and induced traffic on scheme benefits WILL be estimated quantitatively unless there is a compelling reason for not doing so.
• Throughout the Advice there are a number of important recommendations shown highlighted and in bold: if these actions are not followed, analysts will need to provide rigorous justification for the course of action taken.
• Even if induced traffic does not weaken the case for the scheme appreciably, the assessment may be criticised if it cannot demonstrate that the case is robust against possible changes in demand.
• In modelling demand, some segmentation by trip and traveller type is essential: at minimum there should be categorisation by trip purpose (at least home-based work/education, employer's business, and 'other' purposes); some form of distinction between travellers with and without a car available is also very desirable, especially where mode-choice is to be considered.
• The amount of detail required in demand modelling will depend upon the particular application, since the effort and cost involved should be commensurate with the investment being assessed and the scale of its effects. Where a multi-level variable demand model is appropriate, it should include a distribution mechanism, and it will generally be desirable to include other mechanisms which can generate or suppress car trips as congestion reduces or grows.
• The Department's long-established preferred approach to use an incremental rather than an absolute model, unless there are strong reasons for not doing so is reinforced by the above changes.
• Where variable demand modelling is justified, compatibility (convergence) between the assignment and the demand model(s) is very important. To optimise processing time and ensure true converged solutions the travel cost formulations used in both should contain the same ratio of weights of journey time relative to journey distance.
• The sensitivity parameters in the demand mechanisms should use robust calibrated local parameters wherever possible (from existing local models, for example). Failing this the Advice provides illustrative values obtained from a review of current multi-stage demand models.
• The sequence of the distribution and mode split stages in the calculation hierarchy should depend upon the relative strengths of the sensitivity parameters, but trip frequency should always be calculated first and micro time period choice (peak-spreading), if it is to be included, will generally be lowest in the hierarchy. However, the sensitivity parameters should always increase along this sequence from highest to lowest, and this may require different sequences for different categories of travel.
• It is essential to apply "realism testing" to ensure that the model responds rationally and with acceptable elasticities. As a minimum, it is necessary to check the elasticity of demand with respect to car journey time and car fuel costs.
• It is also desirable to apply sensitivity testing to the results of the assessment against variation in those parameters that are uncertain. Generally, in a scheme aimed at congestion relief the net benefit will be reduced by increases in the demand for car travel, so that increasing the sensitivity parameters in the demand mechanisms will test the robustness of the result. If the scheme remains well justified against these higher values then a conclusion that the scheme is beneficial will be robust against the effects of induced traffic.

2. Further Information

The following documents provide information that follows on directly from the key topics covered in this TAG Unit.

3. References

Batley R, T Fowkes,, G Whelan and A Daly.(2001) Models for choice of departure time. European Transport Conference, Seminar on methodological innovations Homerton College, Cambridge Sept 2001.

Bradburn, P and Hyman, G (2002). An econometric investigation of car use in the National Transport Model for Greet Britain. European Transport Conference, September 2002, Seminar on Applied Transport Methods, Homerton College Cambridge 2002.

Department for Transport (2003) Major Scheme Appraisal in Local Transport Plans: Part 3. Available from DfT website as DfT_localtrans_source_504021.doc. October 2003.

DMRB 12.2.2 HMSO February 1997 - withdrawn Summer 2004.

Graham D & S Glaister (2002) Review of Income and Price Elasticites of Demand for Road Traffic. Final report to the DTLR. July 2002.

Hanly M, J Dargay & P Goodwin (2002) Review of Income and Price Elasticities in the Demand for Road traffic. Final Report to DTLR. March 2002.

TRACE (1999). TRACE Final Report, www.transport-research.info/web/projects/project_details.cfm?id=766.

TRL (2004) The Demand for Public Transport: A Practical Guide. TRL Report TRL593. Crowthorne UK.

4. Document Provenance

This Transport Analysis Guidance (TAG) Unit reflects the consultation comments received on the Convergence, Realism & Sensitivity Testing Stage of the draft Variable Demand Modelling Advice produced by TRL in June 2003.

Technical queries and comments on this TAG Unit should be referred to:

Integrated Transport Economics and Appraisal (ITEA) Division
Department for Transport
Zone 3/06, Great Minster House
76 Marsham Street
London SW1P 4DR

E-mail: itea@dft.gsi.gov.uk
Tel: 020 7944 6176
Fax: 020 7944 2198