Neuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Hannover, 2010-02-23

Image Understanding withOrganic Computing

Rolf P. Wurtz

Ruhr-Universitat BochumInstitut fur Neuroinformatik

http://www.neuroinformatik.rub.derolf.wuertz@neuroinformatik.rub.derolf.wuertz@organic-computing.org

OverviewNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

• Introduction

• Problem of image understanding

• Controlled generalization in face recognition

• General object recognition

• Learning of articulated models

• Where to go from here

Complexity problemsNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

• Artificial systems are rapidly getting too complex to understand

• Desirable are complex systems with trivial interfaces

• This requires restrictions on possible behaviors

• This asks for self-organization

Image understanding . . .Neuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

. . . means establishing a symbolic description.

Image understanding . . .Neuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

. . . in the brain is done under additional assumptions.

Image understanding . . .Neuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

. . . requires extensive world knowledge.

Image understanding . . .Neuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

. . . has a local-global problem.

Face recognitionNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

= 6=

Different situations yield very different images.

Invariance problemNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

All the same?

Invariance . . .Neuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

6 6= 6

. . . is task-dependent.

List of tasksNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

• We do need a formal model of images, but we don’t have any

• We do not even have a formalization of the problem

• Required is an imitation of human capability

• Identify constraints of visual data autonomously

• Learn computer vision routines from examples

• Start a positive feedback loop of learning vision

• Control generalization

Controlled generalizationNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

• Neural networks learn complicated functions from examples

• They can generalize, but not always in the desired way

• Visual invariances must be built in explicitly(Neocognitron, Convolutional NN, . . . )

• Exception: Slow feature analysis (Wiskott & Sejnowski, 2002)

• Goal: Learn generalization dimensions from examples!

Bunch GraphsNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

image graph

general facebunch graph Gabor wavelet jet

bunch of jets

a: b: c: Wiskott et al., 1997

Graph similarityNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

• Jet similarity function SJ

• Probe graph P with N nodes Pn

• Gallery graphs Gg with N nodes Gg,n each

grec = arg maxg

1

N

N∑n=1

SJ(Pn, Gg,n) .

Face GraphsNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

Pose VariationNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

PM+45 PM+00 PM−45

Pose VariationNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

Probe Gallery

Similarity?

Gg P

......

Rank CorrelationNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

ModelProbe Gallery

SP SG

π = [7, 3, 9, . . .] γ = [7, 9, 3, . . .]

γ = [2, 1, 35, . . .]

......

......

Neural rank list similarityNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

aj wjE

I

E =

K∑j=1

exp

(−order(aj)

λ

)wj

wj =1

Kexp

(−order(bj)

λ

)

Thorpe et al., 2001

Neural rank list similarityNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

π(m)

wm,g

Ag

wm,g =1

NMexp

(−γg(m)

λ

)Ag =

∑m

exp

(−π(m)

λ

)wm,g

=1

NM

∑m

exp

(−π(m) + γg(m)

λ

)= Sneural(γg, π)

grec = arg maxg

1

N

∑n

Sneural(γg,n, πn)

Muller and Wurtz, ICANN 2009

Face recognition Mainz Hbf.Neuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

CAS-PEAL DatabaseNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

PM+45 FM+00 FM−45 FM−90

PM+00 FD+00 FD−45 FD−90

PM−45 FU+00 FU−45 FU−90

Rank Correlation: ResultsNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

Pose Illumination

Recognition percentage with given situation 99.02 89.01

Percentage of correct situation estimation 99.89 ± 0.09 91.96 ± 0.89

Recognition percentage with automaticallydetermined situation

97.75 ± 0.50 89.97 ± 1.36

Best recognition percentage reported indatabase description

71 51

Rank Correlation: Early stoppingNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100

Rec

ogni

tion

rate

Spike number

poseillumination

Thanks toNeuroinformatik / Ruhr-Universitat Bochum

Rolf P. Wurtz

Image Understanding with Organic Computing Hannover, 2010-02-23

Marco Muller Rank correlation memoryGunter Westphal Object recognitionThomas Walther Body tracking

Manuel Gunter Statistical face recognitionMarkus Lessmann Scene analysisOliver Lomp Neuronal dynamicsMathis Richter Clustering of image patchesGuillermo Donatti Object memory, Neural Map

DFG, EU, NRW Funding

All of you Attention