Master thesis presentation (VU)

76
Safety Risk Assessment for Aircraft Fuel Management Viktor Gregor 10.7.2015

Transcript of Master thesis presentation (VU)

Page 1: Master thesis presentation (VU)

Safety Risk Assessment for Aircraft Fuel Management

Viktor Gregor

10.7.2015

Page 2: Master thesis presentation (VU)

Agenda

• Context of the project

• Introduction to splitting methods

• Modelling methodology

• Introduction to Dynamically coloured Petri nets

• Risk assessment model

• Results of Monte Carlo simulations

• Importance function

• Results of splitting method simulations

• Conclusion and recommendations for further research

2

Page 3: Master thesis presentation (VU)

Introduction Objective and context

• Objective: To develop a risk assessment model for aircraft fuel management

• Context: Fuel management is a crucial part of flight planning.

• Safety regulations ensure that aircraft has reserve fuel.

• Airlines try to minimize the fuel consumption.

• Probability assessment of fuel management related event such as

• Having less than final reserve fuel (FRF) remaining. FRF is the amount of fuel

needed to fly at 450 m for 30 minutes

• Fuel starvation

3

Page 4: Master thesis presentation (VU)

Introduction Examples of fuel-related incidents

• Ryanair in 2010

• After two missed approaches and a

diversion, the aircraft landed at an

alternate airport with less than FRF

remaining.

• Main causes:

• Inadequate decision-making of the crew

• ATC at destination did not provide

suitable information about the wind

4

Alicante airport destination

Valencia airport chosen alternate

Murcia-San Javier airport possible alternate

Page 5: Master thesis presentation (VU)

Introduction Examples of fuel-related incidents

5

• Hapag-Lloyd in 2000

• Aircraft ran out of fuel 20 km before the

airport and had to glide toward the

runway. It landed on the grass 500 m

from the runway. The fuselage was

severely damaged and some passengers

were injured.

• Main causes:

• Malfunction of the landing gear

• Inadequate decision to divert to Vienna

(220 km away) even though Zagreb was

only 75 km away

Vienna airport Chosen diversion

Zagreb airport Possible diversion

Page 6: Master thesis presentation (VU)

Introduction Rare event simulations

• Purpose of risk assessment models

• Determine whether proposed operation is safe

• Find potential risk mitigating measures

• Risk is typically very small. According to European aviation safety agency, the

probability should be

• less than 10−5 per flight hour for major failure events

• less than 10−9 per flight hour for catastrophic failure events

6

Page 7: Master thesis presentation (VU)

Monte Carlo simulation

• Regular Monte Carlo is insufficient for such small probabilities

• Assume we want to estimate 𝛾 = 10−6 and that one simulation run takes 0.01 𝑠

• If we want to have at least 200 hits to the rare event, the expected time of

simulation is 200 ∙ 106 ∙ 0.01 = 2 ∙ 106 𝑠 ≅ 23 days

• Faster algorithms

• Importance sampling: change of underlying probability measure

• Splitting method: simulate under the original probability measure, but prioritize

specific trajectories

7

Page 8: Master thesis presentation (VU)

Splitting method

• Strong Markov process with càdlàg trajectories

• Closed set

• We aim to estimate

8

, 0X X t S t

B S

P ,BT T

inf 0 :

inf 0 :

BT t X t B

T t X t D

Page 9: Master thesis presentation (VU)

Splitting method (2)

• We estimate conditional probabilities of getting from one set to next.

9

0 1 .nB B B B X B Main idea: that has to be crossed by to get to

:

: , 0, ,

Sets are defined using importance function and levels.k

0 n

k k

B h S

L … L = L

B x S h x L k n

3B 2B 1B

B

Page 10: Master thesis presentation (VU)

Splitting method (3)

• Defining

we can express using conditional probabilities.

• Levels have to be chosen such that these conditional probabilities are not too

small to be simulated by regular Monte Carlo.

10

0 ,nA A

0

1 1 0 0

P P P P

P | P | P

n

B n n k

k

n n

T T T T A A

A A A A A

inf 0 : 0, ,,

k k

k k

A T T

T t X t B k n

Page 11: Master thesis presentation (VU)

Splitting method variations

11

• There are several variations of splitting method based on how we determine

the number of “splits” and how we resample the trajectories.

• Fixed splitting: each trajectory that reaches the next level is copied into fixed

number of independent trajectories.

• Fixed effort: number of total trajectories simulated for each level is fixed.

• Fixed number of successes: new trajectories are being simulated until we have a

fixed number of hits to the next level.

Page 12: Master thesis presentation (VU)

Splitting method example

• Then

12

t

t

W

W a > 0 - D < 0

Process is a standard Brownian motion. We want to estimate

the probability that hits level before hitting .

inf 0 : inf 0 :a t tT t W a T t W D Denote and

P a

DT T

a D

Page 13: Master thesis presentation (VU)

Splitting method example (2)

13

610 1 -6

6

1a D 10

1+10 Let and . Then

P

We want to choose levels such that the conditional probabilities

are approximately , where k

k

k+1 k k L

L

A | A 0.1 A T T

0 0 10 , 1, , 6k

kL L k and

1

1

1P | 0.1

1

k

k k

k

LA A

L

Page 14: Master thesis presentation (VU)

Splitting method example (3)

• Expected number of simulations needed to get one hit:

• Naive Monte Carlo: 1 ∙ 1 000 000

• Splitting method: 6 ∙ 10

• Assuming that one simulation run takes 0.01 seconds and we want 200 hits:

• Naive Monte Carlo: 1 000 000 ∙ 200 ∙ 0.01 seconds ≅ 23 days

• Splitting method: 60 ∙ 200 ∙ 0.01 seconds = 2 minutes

14

Page 15: Master thesis presentation (VU)

Design of the risk assessment model

• TOPAZ (Traffic Organization and Perturbation AnalyZer) safety risk assessment

cycle has been used to develop the model

15

Page 16: Master thesis presentation (VU)

Identify objective and determine operation

• Scope of the assessment is limited to risk of attaining low usable fuel during

commercial air transport operations with turbine engine aeroplanes.

• That includes:

• Pre-flight planning

• In-flight fuel management

• Human roles included in the assessment:

• Crew of the aircraft

• Airline operational control (AOC)

• Air traffic control (ATC)

16

Page 17: Master thesis presentation (VU)

Determine operation

• Pre-flight fuel planning:

• Optimal route and altitude

• Fuel requirements:

• Taxi fuel

• Trip fuel

• Contingency fuel

• Alternate fuel

• Final reserve fuel (FRF)

• Additional fuel

• Extra fuel

17

• In-flight fuel management:

• In-flight fuel checks

• Control of the planning

assumptions

• Replanning and diverting

Page 18: Master thesis presentation (VU)

Identify hazards and construct scenarios

• List of 150 hazards categorized into clusters

18

D1 – Resolution hazards

Landing with less than

final reserve fuelFuel starvation

Fuel shortage

A – Fuel consumption B – Flight length C – Unavailable fuel

C1 – Loss of fuel C2 – Inability to use fuel C3 – Low fuel intake

Page 19: Master thesis presentation (VU)

Agent-based model

• Agent-based dynamic risk modelling

• Agent: an autonomous entity which interacts with other agents to exchange

information.

• Agents included in current version of the model:

1. Environment

2. Airports

3. Airline operational control

4. Aircraft

5. Crew of the aircraft

19

Page 20: Master thesis presentation (VU)

Dynamically coloured Petri nets

• Model is constructed using dynamically coloured Petri nets

• Dynamically coloured Petri nets (DCPN) are equivalent to a piecewise

deterministic Markov process (PDP) in a sense that under some conditions

there exists:

• A one-to-one mapping from PDP to DCPN

• An into mapping from DCPN to PDP

20

Page 21: Master thesis presentation (VU)

Dynamically coloured Petri nets example

• Petri net consists of:

• Places

• Tokens

• Transitions

• Immediate

• Delay

• Guard

• Arcs

• Ordinary

• Enabling

21

GStartMain

processG End

Background

process

Nominal

mode

D

Non-

nominal

mode

D

Page 22: Master thesis presentation (VU)

Dynamically coloured Petri nets example

22

GStartMain

processG End

Background

process

Nominal

mode

D

Non-

nominal

mode

D

• Petri net consists of:

• Places

• Tokens

• Transitions

• Immediate

• Delay

• Guard

• Arcs

• Ordinary

• Enabling

Page 23: Master thesis presentation (VU)

Dynamically coloured Petri nets example

23

• Petri net consists of:

• Places

• Tokens

• Transitions

• Immediate

• Delay

• Guard

• Arcs

• Ordinary

• Enabling

GStartMain

processG End

Background

process

Nominal

mode

D

Non-

nominal

mode

D

Page 24: Master thesis presentation (VU)

Overview of the agents

24

Environment Airports

Airline operational

control

AircraftCrew of the aircraft

Page 25: Master thesis presentation (VU)

Environment agent

• Airspace is divided into 𝑁 × 𝑀 sectors. Each sector has information about

• Wind speed and direction

• Sector availability

• Wind and availability changes at random intervals

• Local Petri net EN

25

EN_P1

D1

D2

EN

Page 26: Master thesis presentation (VU)

Airports agent

• Information about all airports considered in the model

• Location of the airport

• Taxi time

• Parameters regarding holding and missed approach

• Local Petri net AP

26

AP_P1

I1_i

I2_i

AP

Page 27: Master thesis presentation (VU)

Airline operational control agent

• AOC constructs the flight plan and calculates fuel requirements

• Planned trajectory

• Planned fuel consumption

• Local Petri net AO

27

AO_P1 I1 AO_P2

AO

Page 28: Master thesis presentation (VU)

Aircraft agent

• Aircraft agent consists of three local Petri nets:

• Aircraft characteristics AC_CH: includes all parameters of the aircraft

• Aircraft fuel system AC_FS: simulates the fuel consumption

• Aircraft evolution AC_EV: simulates the actual flight from taxi-out to taxi-in

28

AC_CH_P1

AC_CH

AC_FS_P1 G1

AC_FS

Page 29: Master thesis presentation (VU)

Aircraft evolution local Petri net AC_EV

29

G1

P1

Taxi out

P2

Climb

P4

Descent

P3

Cruise

P5

Taxi in

P6

Missed

approach

P7

Hold

G2

G4 G5

G8G7 G9 G11

G10

G3

P0

Start

P8

End

G0

G

G6

22 – 28

G

12 – 16

G

17 – 21

I

29 – 34

AC_EV

Page 30: Master thesis presentation (VU)

Crew agent

• Crew agent consists of two local Petri nets:

• Crew planning CR_PL: includes the decisions made before the flight

• Crew situation awareness CR_SA: includes the decisions made during the flight,

the intentions of the crew and situation awareness of the crew

30

CR_PL_P1 G1

CR_PL

CR_SA_P1 G3I1

I2

G4G5G6

CR_SA_P2

CR_SA

Page 31: Master thesis presentation (VU)

Output of the model altitude and fuel consumption over time

Altitude of aircraft Fuel consumption

31

Page 32: Master thesis presentation (VU)

Output of the model altitude and fuel consumption over time

Altitude of aircraft Fuel consumption

32

Page 33: Master thesis presentation (VU)

Output of the model determining route and diversions

33

A

C

B

Page 34: Master thesis presentation (VU)

Output of the model determining route and diversions

34

A

C

B

Page 35: Master thesis presentation (VU)

Output of the model determining route and diversions

35

A

C

B

Page 36: Master thesis presentation (VU)

Output of the model determining route and diversions

36

A

C

B

Page 37: Master thesis presentation (VU)

Output of the model determining route and diversions

37

A

C

B

Page 38: Master thesis presentation (VU)

Output of the model determining route and diversions

38

A

C

B

Page 39: Master thesis presentation (VU)

Monte Carlo simulation Results

• 6 simulations of 50 000 realizations

• Histogram of remaining fuel after the flight

39

Page 40: Master thesis presentation (VU)

Monte Carlo simulation Results (2)

• Number of flights with less than final reserve fuel (FRF) left

40

number of observations 5 12 8 9 4 4

probability estimate

1.00 ∙ 10−4 2.41 ∙ 10−4 1.60 ∙ 10−4 1.80 ∙ 10−4 8.02 ∙ 10−5 8.02 ∙ 10−5

• All results combined

number of observations

probability estimate

relative error

42 1.40 ∙ 10−4 0.215

Page 41: Master thesis presentation (VU)

Monte Carlo simulation Results (3)

• Holding has significant effect on the amount of remaining fuel.

• More than 97% of observations had positive holding time.

41

Page 42: Master thesis presentation (VU)

Monte Carlo simulation Results (4)

• Effect of different parameter values on the amount of remaining fuel

• Higher variability of wind direction

42

parameter original values changed values

𝜎𝜑 𝜋

32

𝜋

8

Page 43: Master thesis presentation (VU)

Monte Carlo simulation Results (4)

• Effect of different parameter values on the amount of remaining fuel

• Higher variability of wind direction

43

parameter original values changed values

𝜎𝜑 𝜋

32

𝜋

8

Page 44: Master thesis presentation (VU)

Monte Carlo simulation Results (5)

• Higher variability of wind direction

• Number of flights with less than final reserve fuel (FRF) left

44

number of observations 18 15 11 13 8 10

probability estimate

3.61 ∙ 10−4 3.01 ∙ 10−4 2.21 ∙ 10−4 2.61 ∙ 10−4 1.60 ∙ 10−4 2.01 ∙ 10−4

• All results combined

number of observations

probability estimate

relative error

75 2.51 ∙ 10−4 0.118

Original result

42 1.40 ∙ 10−4 0.215

Page 45: Master thesis presentation (VU)

Monte Carlo simulation Results (6)

• Effect of different parameter values on the amount of remaining fuel

• Higher probability of holding

45

parameter original values changed values

𝑝ℎ𝑜𝑙𝑑 0.05 0.1

Page 46: Master thesis presentation (VU)

Monte Carlo simulation Results (6)

• Effect of different parameter values on the amount of remaining fuel

• Higher probability of holding

46

parameter original values changed values

𝑝ℎ𝑜𝑙𝑑 0.05 0.1

Page 47: Master thesis presentation (VU)

Monte Carlo simulation Results (6)

• Effect of different parameter values on the amount of remaining fuel

• Higher probability of holding

47

parameter original values changed values

𝑝ℎ𝑜𝑙𝑑 0.05 0.1

Page 48: Master thesis presentation (VU)

Monte Carlo simulation Results (7)

• Higher probability of holding

• Number of flights with less than final reserve fuel (FRF) left

48

number of observations 17 11 18 12 16 12

probability estimate

3.41 ∙ 10−4 2.21 ∙ 10−4 3.61 ∙ 10−4 2.41 ∙ 10−4 3.21 ∙ 10−4 2.41 ∙ 10−4

• All results combined

number of observations

probability estimate

relative error

86 2.87 ∙ 10−4 0.086

Original result

42 1.40 ∙ 10−4 0.215

Page 49: Master thesis presentation (VU)

Monte Carlo simulation Results (8)

• The dependence of the probability on the threshold for remaining fuel.

• Original parameters

49

Page 50: Master thesis presentation (VU)

Monte Carlo simulation Results (9)

• The dependence of the probability on the threshold for remaining fuel.

• Higher wind variability

50

Page 51: Master thesis presentation (VU)

Monte Carlo simulation Results (10)

• The dependence of the probability on the threshold for remaining fuel.

• Higher probability of holding

51

Page 52: Master thesis presentation (VU)

Splitting method simulation Importance function and levels

• We have piecewise deterministic Markov process

• We need to determine the importance function

and levels .

1. Naive approach: During simulations, we want to reach very low fuel levels.

Importance function:

52

, 0X X t S t

:h S

0 nL L

. is the amount of fuel left at state f fm m x x

1 ff x m

does not work!1f

Page 53: Master thesis presentation (VU)

Splitting method simulation Importance function and levels (2)

2. We want to reach state, where we burn more fuel than was expected. We will

use function:

53

0

.

is the distance of the aircraft from the destination at state .

is the distance from departure airport to the destination.

is the planned amount of fuel left at state

is the

fAO fAO

f f

d d x x

d

m m x x

m m x

. actual amount of fuel left at state x

2

0 0

fAO f

fAO fAO

m d mf x

m d m

Page 54: Master thesis presentation (VU)

Splitting method simulation Importance function and levels (3)

• Usual development of function along the flight

54

2f

Page 55: Master thesis presentation (VU)

Splitting method simulation Importance function and levels (3)

• Usual development of function along the flight

55

2f

Page 56: Master thesis presentation (VU)

Splitting method simulation Importance function and levels (4)

• Horizontal levels are not effective.

56

Page 57: Master thesis presentation (VU)

Splitting method simulation Importance function and levels (5)

• We set a “distance condition” for each level.

57

Page 58: Master thesis presentation (VU)

Splitting method simulation Importance function and levels (6)

• This is not correct because it does not generate a decreasing sequence of sets

58

1 nB B

Page 59: Master thesis presentation (VU)

Splitting method simulation Importance function and levels (7)

• These levels produce satisfactory results.

59

Page 60: Master thesis presentation (VU)

Splitting method simulation results Probability of reaching less than FRF

• We used the splitting method with fixed number of successes.

• Number of successes was set to 200 and we set seven levels.

60

probability estimate

relative error

1.599 ∙ 10−4 0.152

• Result is similar to one got by naive Monte Carlo.

Naive Monte Carlo: Computation time to get on average 7 hits

Splitting method: Computation time to get 200 hits

8 hours 1 hour

• Results from 60 simulations:

Page 61: Master thesis presentation (VU)

Splitting method simulation results Probability of fuel starvation

• We used the splitting method with combination of fixed number of successes

and fixed effort.

• Number of successes was set to 500 and we set nine levels.

• Results from 43 simulations:

61

probability estimate

relative error

3.647 ∙ 10−8 0.273

• Result is in line with the exponential extrapolation made from results of

naive Monte Carlo.

Page 62: Master thesis presentation (VU)

Splitting method simulation results relative error

• In an article introducing the fixed number of successes variation (Amrein,

Künsch 2010) a simplifying assumption is made that

62

1 1 1 1 1| , , , , ,k k k k k k kP A A T X p T X k

1 1where are entrance values and are entrance times.k kX T

• This assumption is not always true in our simulation. For example, probability

of hitting next level is different if we hit a level during usual landing than if

we hit it during performing missed approach.

Page 63: Master thesis presentation (VU)

Conclusions and recommendations for further research

• We developed the first version of a dynamic risk assessment model for fuel

management. Using splitting method, the model produces reasonable results

for probabilities of reaching very low fuel levels.

• The splitting method used is also referred to as Interacting particle system

algorithm (IPS). There are more sophisticated extensions that can lead to

smaller relative error:

• Hybrid interacting particle system algorithm (HIPS)

• Hierarchical hybrid interacting particle system algorithm (HHIPS)

63

Page 64: Master thesis presentation (VU)

Recommendations for further research Petri net model

• Inclusion of an agent for ATC

• More hazards included in the model

• More complex decision making

• More sophisticated model of wind and wind prediction

• Correlation between adjacent sectors

• Inclusion of hazards related to fuel system (e.g. fuel leakage)

• More complex decision making of crew

• Inclusion of several aircraft for more realistic modelling of high traffic

64

Page 65: Master thesis presentation (VU)

65

Thank you for your attention.

Page 66: Master thesis presentation (VU)

Appendix Interactions of EN with other local Petri nets

66

EN_P1

D1

D2

AO_P1

I

AO_P2

AC_FS_P1G1

AC_EV

CR_SA

Page 67: Master thesis presentation (VU)

Appendix Interactions of AP with other local Petri nets

67

AP_P1

I1_i

AO_P1

I

AO_P2

AC_EV

CR_SA

IPN1_AP_i

I2_iIPN2_AP_i

Page 68: Master thesis presentation (VU)

Appendix Interaction Petri nets of airports

• Interaction Petri nets trigger the decision of the airport whether the aircraft will

be sent to holding.

68

IPN1_AP_i AP_P1I1_i

G3

G4

G8

AC_EV

IPN2_AP_i AP_P1I2_iG9

AC_EV

Page 69: Master thesis presentation (VU)

Appendix Interactions of AO with other local Petri nets

69

AO_P1

I1

AO_P2

EN_P1

AP_P1

AC_CH_P1

AC_EV

CR_PL_P1

G AC_FS_P1

CR_SA

Page 70: Master thesis presentation (VU)

Appendix Interactions of AC_CH with other local Petri nets

70

AC_CH_P1

AO_P1

I

AO_P2

AC_FS_P1G

CR_SA

CR_PL_P1G

Page 71: Master thesis presentation (VU)

Appendix Interactions of AC_FS with other local Petri nets

71

AC_FS_P1G1EN_P1

AC_CH_P1

CR_PL_P1 G

AC_EV

CR_SA

Page 72: Master thesis presentation (VU)

Appendix Interactions of AC_EV with other local Petri nets

72

EN_P1

AO_P2

AP_P1

IPN_AC_EV

AC_EV

CR_SA

IPN2_AP_i IPN_CR_SA

AC_FS_P1G

IPN1_AP_i

Page 73: Master thesis presentation (VU)

Appendix Interaction Petri nets of aircraft evolution

• Interaction Petri net triggers the change of flight route updated by the crew

agent.

73

IPN_AC_EV

I1

G7

CR_SA

I

AC_EV

29 – 34

Page 74: Master thesis presentation (VU)

Appendix Interactions of CR_PL with other local Petri nets

74

CR_PL_P1

G1 AC_FS_P1

AO_P2

AC_CH_P1

Page 75: Master thesis presentation (VU)

Appendix Interactions of CR_SA with other local Petri nets

75

EN_P1

AO_P2

AP_P1

CR_SA

IPN_AC_EV

AC_CH_P1

AC_FS_P1

AC_EV

IPN_CR_SA

Page 76: Master thesis presentation (VU)

Appendix Interaction Petri net of crew SA

• Interaction Petri nets trigger the decision of the crew whether they will perform

a missed approach before landing.

76

IPN_CR_SA I2

CR_SA

G3

G4

G8

AC_EV

G11