VI Escola de Sistemas Embarcados - INFcaco/ESSE2016/ESSE2016_Nacif.pdf · VI Escola de Sistemas...

Lucas Bragança da Silva

Fredy Alves

José Nacif ( Apresentador )

Ricardo Ferreira ( Apresentador )

Universidade Federal de Viçosa

Finantial Support: Intel Brasil, Intel Labs, Capes, Cnpq, Fapemig

VI Escola de Sistemas Embarcados

ESSE 2016VI Brazilian Symposium on Computing Systems Engineering

CPU/FPGA Heterogeneous

Architectures

“CGRA HARP”

VI Brazilian Symposium on Computing Systems Engineering, November 2016

Outline

• Motivation• FPGA and CPU • OpenCL and FPGA accelerators• HARP Platform

• HARP Layers• Demo• HARP CGRA

“CGRA HARP”

Motivation

“CGRA HARP”

Motivation

Moore Law continues….

Single Thread

Frequency

“CGRA HARP”

Motivation

Moore Law continues….

multiple

“CGRA HARP”

MotivationIoT and Cloud Computing• “Coherently-attached FPGA accelerator for Xeon processors in the

datacenter which is estimated to have a $1B market opportunity by 2020”

- Prabhat K. Gupta - General Manager of Xeon+FPGA Product at Intel Corporation

• Microsoft Cataput: layer of reconfigurable logic (FPGAs) between the network switchesand the servers (enabling the FPGAs to communicate directly, at datacenter scale) - IEEE Micro2016 - “A Cloud-Scale Acceleration Architecture”

• Baidu, Inc. (NASDAQ: BIDU), the leading Chinese language Internet search provider• Accelerators = greater throughput at low latency while retaining practical power levels.• 10-20X performance/watt improvement.• Baidu-optimized FPGA platforms are tuned for machine learning applications such as image

and speech recognition.

“CGRA HARP”

Motivation FPGA

“CGRA HARP”

Motivation FPGA

“CGRA HARP”

Motivation FPGA

“CGRA HARP”

Motivation FPGA

“CGRA HARP”

Motivation FPGA

“CGRA HARP”

FPGA is scalable !

“CGRA HARP”

• Scalable• Energy Efficiency• Parallel and distributed computing• Temporal and Spatial Parallelism• From low cost embedded to high performance

“CGRA HARP”

FPGAs andTools

Hardware Description

Languages

Compilers

High Level Synthesis

General Purpose

“CGRA HARP”

FPGAs andTools

Specific tools for

specific applications

CPU and FPGAs

• Heterogeneous applications and Heterogeneous hardware

• Real World• HARP - Intel/Altera Platform

• Microsoft Cataput

FPL 2016 - PK Gupta - Intel

Accelerating DataCenter Workloads

Microsoft Cataput v2

172.6K ALMs ,4 GB DDR3

RoundTrip - 250,000 machines

in 20 microseconds,

Microsoft Cataput v2

• Microsoft's FPGA Translates Wikipedia in less than a Tenth of a Second

• FPGA network - breaking the “chicken and egg”• accelerators cannot be added until enough applications need them,

but applications will not rely upon the accelerators until they are present in the infrastructure.

• By decoupling the servers and FPGAs, software services that demand more FPGA capacity

Instructions

Temporal and

Spatial Parallelism

Temporal and

Spatial Parallelism

OpenCL

OpenCL example

__attribute__(num_compute_units(4,4))

kernel void PE() {

How to build a systolic computer…..

kernel void PE() {

OpenCL example

kernel void PE() {

row = get_compute_id(0);

col = get_compute_id(1);

OpenCL example

channel float4 ch_bottom[4];

PE() {

float4 a,b;

if (row == 0)

a = read_channel

(ch_bottom[col]); PE

OpenCL example channel float4 ch_bottom[4];

channel float4 ch_PE_col[4][4];

…PE() {

float4 a,b;

if (row == 0)

a = read_channel(ch_bottom[col]);

a = read_channel(ch_PE_col[row-1][col])

Coarse Grained Reconfigurable Array CGRA vs FPGA

Huge Bitstream…..

fine grained Bitstream

CGRA as a virtual layer Small Bitstream

HARP - Legal Disclaimer

Copyright (C) 2008-2016 Intel Corporation All Rights Reserved.The source code contained or described herein and all documents relatedto the source code ("Material") are owned by Intel Corporation or itssuppliers or licensors. Title to the Material remains with Intel Corporationor its suppliers and licensors. The Material contains trade secrets andproprietary and confidential information of Intel or its suppliers andlicensors. The Material is protected by worldwide copyright and tradesecret laws and treaty provisions. No part of the Material may be copied,reproduced, modified, published, uploaded, posted, transmitted,distributed, or disclosed in any way without Intel's prior express writtenpermission.

HARP Prototype Xeon+FPGA* system disclaimerThis talk is about prototype hardware and software which has been madeavailable to universities in the HARP program.

Details of production Xeon+FPGA systems will be made available at a laterdate

Results and details in this presentation were generated using pre-production hardware and software, and may not reflect production or futuresystems

HARP Accelerating Workloads using Xeon and coherently attached FPGA in-socket

QPI1 6 GB/s

Heterogeneous architecture with homogenous platform support1QuickPath Interconnect

HARP-1 – Development platform

• 96 GB RAM

• Xeon 10 cores

• FPGA Stratix V- 622K LUTs

- 1M Registers

- 2.5K Memory Modules (M20K)

- 512 DSPs

Stratix

HARP-1 – USB Programmer

HARP HDL Programming

HARP – General Architecture

HARP - Accelerator Abstraction Layer (AAL)

• Set of software tools for development and deployment of systems composed by asymmetric computing resources

• CPUs, GPUs, FPGAs as a server

• An application uses the server by requesting resources

• Resource manager• Ensuring exclusiveness of the use of a resource.

• Service-oriented and object-oriented • Interface definitions, attributes and the objects which implements those interfaces.

• Service-Oriented Architecture• Service: Encapsulation of functionality which consumes computing resources

• Registrar: Registers services and APIs, used to locate and acquire service interfaces.

• Client: Executable that uses a service by acquiring API from registrar.

HARP – AAL Object Communications

• AAL uses an asynchronous communication, It returns to the application while the requested service execute in parallel.

HARP – Services, Interfaces, Composition

• Client accesses the service through virtual interfaces published on the Registrar which does not expose the implementation.

• Component objects implement the interface.

class IMyInterface{public:virtual doThis(void)=0;virtual doThat(void)=0;virtual

~IMyInterface(){}};

HARP – Abstraction, Resource Management

• AAL abstracts service instantiation from the application.

• Services can be created dynamically.

• When multiple implementations of a service are available in one or more compute resources, AAL returns the most suitable one.

• AAL Resource Manager controls the allocation and provisioning of compute resources to services.

• Resource management is important for precious and shared

resources such as accelerators on FPGAs.

HARP – AAL Service Broker and Registrar

• Service Broker gets the information required to instantiate a service from the Registrar.

• Service libraries are loadable software such as DLLs.

• Client 1 consults Service Broker for Service Compute.

• Service Broker obtains data record describing Service Compute from Service Registrar.

• Service broker consults Resource Manager which consults implementations and computing resources.

• Resource Manager returns information to allow Broker to load service package.

• Service broker calls Service factory to instantiate it.

HARP – Core Cache Interface (CCI)

• Interface between AFU and QPI

- Read and write requests to the system coherent memory.

- Coherent memory is mapped to CPU DRAM.

• FPGA implements Intel QPI

- Processor uses QPI to access the system cache.

6 GB/s

Line = 64B

HARP – Core Cache Interface (CCI)

• Accelerated Function Units• Accelerates an application kernel, in the FPGA.

• Blue dotted box is the multiprocessor boundary.

• Red dotted box is the Cache access domain

6 GB/s

Line = 64B

HARP – Interface Definitions: Attach points

• QPI-FPGA implements the Caching and Configuration agents.

- The caching agent assures memory coherence.

- The Configuration Agent receives and handles read and write cycles from processor.

- System Protocol Layer (Virtual Address Translation)

• Processor-FPGA is RX.

• FPGA-Processor is TX.

• Designed to accept one read and write per clock cycle.

• AFU with CCI-E connected via SPL2, ordered read responses, writes out of order.

• SPL2: up to 2GB pinned virtual address space to an AFU.

• AFU connected via CCI Standard (CCI-S) or CCI Extended (CCI-E).

• CCI-S uses physical addressing and out of order responses.

• CCI-E uses virtual addressing.

• Intel provides SPL2 IP to translate virtual to physical addresses.

• AFU connected to SPL2 via CCI-E.

“CGRA HARP”

Intel HARP

“Hello World”

Example.

Are you ready???

HARP – Accelerator “Hello World”

• AFU capable of adding two CPU memory values- SPL2 RTL for address translation- SW application (C++) & AFU RTL (Verilog)

• In this Example we will be demonstrating the use of:- AAL Runtime- Service API- Example AFU to build an user AFU.

AAL Application code: Run method

m_runtimClient->getRuntime()->allocService(dynamic_cast<IBase

*>(this), Manifest);

m_Sem.Wait();

if(0 == m_Result){

MSG("Running Test");

btVirtAddr pWSUsrVirt = m_pWkspcVirt; // Address of

Workspace

const btWSSize WSLen = m_WkspcSize; // Length of

workspace

INFO("Allocated " << WSLen << "-byte Workspace at virtual

address " << std::hex << (void *)pWSUsrVirt);

// Number of bytes in each of the source and destination buffers

(4 MiB in this case)

btUnsigned32bitInt a_num_bytes= (btUnsigned32bitInt) ((WSLen -

sizeof(VAFU2_CNTXT)) / 2);

• Allocates services, workspace in Device

Status Memory (DSM) using allocService().

• If service is successfully allocated, runs the

• Get the address and the length of the

workspace.

• Defines the size of the source and

destination buffers in bytes.

btUnsigned32bitInt a_num_cl = a_num_bytes / CL(1); //

number of cache lines in buffer

// VAFU Context is at the beginning of the buffer

VAFU2_CNTXT *pVAFU2_cntxt = reinterpret_cast<VAFU2_CNTXT

*>(pWSUsrVirt);

// The source buffer is right after the VAFU Context

btVirtAddr pSource = pWSUsrVirt + sizeof(VAFU2_CNTXT);

// The destination buffer is right after the source buffer

btVirtAddr pDest = pSource + a_num_bytes;

• Defines the number of cache lines on each

buffer in a_num_cl.

• Get the pointer to afu context

(pVAFU2_cntxt), to source buffer

(pSource) and destiny buffer (pDest).

• pDest is pSource plus the size of pSource

in bytes (a_num_bytes).

// Initialize the command buffer

::memset(pVAFU2_cntxt, 0, sizeof(VAFU2_CNTXT));

pVAFU2_cntxt->num_cl = a_num_cl;

pVAFU2_cntxt->pSource = pSource;

pVAFU2_cntxt->pDest = pDest;

INFO("Starting SPL Transaction with Workspace");

m_Sem.Wait();

int num1 = 3;

int num2 = 2;

int *inputs_ADD = (int*)malloc(sizeof(int)*2);

volatile int *addIn = (int*)pSource;

inputs_ADD[0] = numa;

inputs_ADD[1] = numb;

memcpy((void*)addIn, inputs_ADD, sizeof(int)*2);

m_SPLService->StartTransactionContext(TransactionID(), pWSUsrVirt, 100);

• Initialize and copies the AFU

• defines two numbers (numa, numb) and

copy them to the source buffer

• Starts the transaction by using

StartTransactionContext, this enables the

start signal on the AFU and resets it.

• Initialize and copies the AFU context to the

context pointer.

• defines two numbers (numa, numb) and

copy them to the source buffer using

memcpy.

• Starts the transaction by using

StartTransactionContext, this enables the

start signal on the AFU and resets it.

• The AFU writes its AFU_ID to DSM.

• The AFU will be running after the CPU

reads the AFU_ID from the DSM.

// Wait for SPL VAFU to finish code

volatile bt32bitInt done = pVAFU2_cntxt->Status &

VAFU2_CNTXT_STATUS_DONE;

while (!done && --count) {

SleepMilli( delay );

done = pVAFU2_cntxt->Status & VAFU2_CNTXT_STATUS_DONE;

if ( !done ) {

// must have dropped out of loop due to count -- never

saw update

ERR("AFU never signaled it was done. Timing out anyway.

Results may be strange.\n");

int *pu32 = reinterpret_cast<int*>(&pDestCL[0]);

for(int i = 0;i< results_num;i++){

cout << *pu32 << "\n";

++pu32;

• Gets reference to AFU DONE signal.

• Wait for done to be set to 1.

• If AFU do not answer before the limit time,

prints and error message.

• If AFU answers in time, gets the reference

to destiny buffer and print the results.

//Issue Stop Transaction and wait for OnTransactionStopped

INFO("Stopping SPL Transaction");

m_SPLService->StopTransactionContext(TransactionID());

m_Sem.Wait();

// Clean up and exit

INFO("Workspace verification complete, freeing workspace.");

m_SPLService->WorkspaceFree(m_pWkspcVirt, TransactionID());

m_Sem.Wait();

m_runtimClient->end();

// while(1){}

return m_Result;

• After transaction is done, it stops the

transaction.

• By stopping, it resets the AFU and set start

signal to 0.

• Frees the workspace.

• Based on the Sudoku example.

• SPL RTL: Provided by Intel

• AFU RTL

The AFU

- afu_user: implements

communication interface

with the SPL module.

module afu_user #(CACHE_WIDTH = 512)

input clk,

input reset_n,

// Read Request

output [ADDR_LMT-1:0] rd_req_addr,

output [MDATA-1:0] rd_req_mdata,

output reg rd_req_en,

input rd_req_almostfull,

// Read Response

input rd_rsp_valid,

input [MDATA-1:0] rd_rsp_mdata,

input [CACHE_WIDTH-1:0] rd_rsp_data,

• CACHE_WIDTH is the size of the cache

line in bits.

• SW application starts the transaction

(Reset).

• The read request signals are used to

request a read from the source buffer.

• The read response is the read data

reponse. in this case we only use

rd_rsp_data which has the cache line for

the last read request.

The AFU USER Interface

// Write Request

output [ADDR_LMT-1:0] wr_req_addr,

output [MDATA-1:0] wr_req_mdata,

output [CACHE_WIDTH-1:0] wr_req_data,

output reg wr_req_en,

input wr_req_almostfull,

// Write Response

input wr_rsp0_valid,

input [MDATA-1:0] wr_rsp0_mdata,

input wr_rsp1_valid,

input [MDATA-1:0] wr_rsp1_mdata,

// Start input signal

input start,

// Done output signal

output reg done,

// Control info from software

input [511:0] afu_context);

• The write request signals have the same

set of signals as the read request for cache

write operations.

• wr_rsp1_valid is used to identify when the

writing process finishes.

• “start” is the signal set by the CPU in the

AFU to start a transaction.

• “done” is the signal sent to the CPU to

indicate that the transaction processing is

The AFU USER Interface

• Read data from the source buffer, process (AFU), write the results back to the destination buffer.

The AFU Control States

FSM_IDLE: begin

if(start) begin

fsm_ns = FSM_RD_REQ;

• Waits for start signal to be set to one by

the CPU.

• Changes to FSM_RD_REQ to start reading

process from source buffer.

The AFU Control States: FSM_IDLE

FSM_RD_REQ: begin

// If there's no more data to copy

if(addr_cnt >= num_clines)

fsm_ns = FSM_RUN_ADD;

addr_cnt_clr = 1'b1;

// There's more data to copy

else begin

// Issue rd_req

if(!rd_req_almostfull) begin

rd_req_en = 1'b1;

fsm_ns = FSM_RD_RSP;

• addr_cnt keeps track of which line is being

• if addr_cnt is more than the number of

lines to be read (num_clines), change to

state to run user AFU.

• if addr_cnt is less than the number of lines

to be read and the read buffer is not full,

sends a read request to SPL (rd_req_en)

and changes to state to wait for read

response.

The AFU Control States: FSM_RD_REQ

always@(posedge clk)

if(rd_rsp_valid)

case(addr_cnt)

inputs_add <= rd_rsp_data;

endcase // case (addr_cnt)

end // if (rd_rsp_valid)

end // always@ (posedge clk)

adder add0(

.clk(clk),

.start(start),

.numA(inputs_add[31:0]),

.numB(inputs_add[63:32]),

.result(w_outGrid),

.done(w_done)

• This always block waits for a response

(when rd_rsp_valid is 1) and then save the

data from the source buffer (rd_rsp_data),

in this case to inputs_add.

• inputs_add is connected to the input of the

user AFU which in this case is adder.

The AFU Control States: FSM_RD_REQ

FSM_RD_RSP:

// Receive rd_rsp, put read

data into data_buf

if(rd_rsp_valid)

addr_cnt_inc = 1'b1;

fsm_ns = FSM_RD_REQ;

• Waits for response.

• addr_cnt_inc is set to one, this increases

the addr_cnt by 1 which means that the

next line in source buffer will be read.

• Goes back to FSM_RD_REQ.

The AFU Control States: FSM_RD_RSP

// --- Address counter

reg [31:0] addr_cnt;

always @ (posedge clk) begin

if(!reset_n)

addr_cnt <= 0;

if(addr_cnt_inc)

addr_cnt <= addr_cnt + 1;

else if(addr_cnt_clr)

addr_cnt <= 'd0;

• This always block is responsible for

controlling the changes in addr_cnt.

• When addr_cnt_inc is set to 1, increase it

by 1 to go to next buffer line.

• When addr_cnt_clr is set to 1, clears

addr_cnt.

The AFU Control States: FSM_RD_RSP

FSM_RUN_ADD:

t_start = 1'b1;

fsm_ns = FSM_WAIT_ADD;

n_cnt = 'd0;

adder add0(

.clk(clk),

.start(t_start),

.result(w_outGrid),

.done(w_done)

• Sets t_start connected to the adder to 1.

• Adder starts.

• Goes to state to wait adder to finish.

The AFU Control States: FSM_RUN_ADD

FSM_WAIT_ADD:

if(w_done | w_error)

fsm_ns = FSM_WR_REQ;

adder add0(

.clk(clk),

.start(t_start),

.result(w_outGrid),

.done(w_done)

• Waits for w_done wire connected to done

signal on the adder to be set to one

meaning that the adder processing

finished.

• When finished, goes to state to start writing

results to destination buffer.

The AFU Control States: FSM_WAIT_ADD

FSM_WR_REQ:

if(addr_cnt >= num_clines)

fsm_ns = FSM_DONE;

else if(!wr_req_almostfull)

wr_req_en = 1'b1; // issue wr_req

fsm_ns = FSM_WR_RSP;

• Requests data to be written to destiny

buffer is the output of the adder.

The AFU Control States: FSM_WR_REQ

FSM_WR_RSP:

if(wr_rsp0_valid | wr_rsp1_valid)

fsm_ns = FSM_WR_REQ;

addr_cnt_inc = 1'b1; // address counter ++

• The data to be written to destination buffer

is the output of the adder.

The AFU Control States: FSM_WR_RSP

FSM_DONE:

done = 1'b1;

fsm_ns = FSM_DONE;

• Sets done to one which finishes the

transaction, stops the SPL and sends done

signal to the CPU.

The AFU Control States: FSM_DONE

Collision detection

algorithm

• Detects collisions between rigid bodies

in a space and calculate the results for

these collision.

• Used in a wide variety of applications

such as games, simulations and

robotics.

• Implemented in engines. In our case we

integrate the HARP platform with ODE

(Open Dynamics Engine), an open-

source engine.

Case study:

spheres collision detection

• Inputs: Position, speed and

form of bodies in space.

• Outputs: Contact points

(Potential, fake and true).

Collision results (New

position for spheres).

In-game gameplay from the game Besiege.

The system is composed by:

• The ODE application integrated

with AAL.

• The FPGA with the collision

detection AFU and SPL for

address translation.

• Source buffer to hold input data

for AFU.

• Destination buffer to hold the

collision detection results for the

application.

For each simulation step

• The CPU sends the collisions

data to the Source Buffer.

• CPU sends start and reset signal

to SPL which propagates to the

• The AFU sends its AFU ID to the

SRC Buffer which is read by the

CPU indicating that the AFU

started.

• After it finishes processing the

collisions, the AFU sends the

results to the destination Buffer.

• AFU indicates to CPU that it is

done processing the transaction.

• CPU retrieves results from

destination buffer.

Two application examples

FPGA Board Evaluation - DAC,2016

FCCM,2016

DNA accelerator

short sequence

Harp/Intel

FCCM,2016

DNA accelerator

short sequence

Harp/Intel

FCCM,2016

Inside a PE

FCCM,2016

Applications mapped on HARP

• “Runtime Parameterizable Regular Expression Operators for Databases”

• tradeoff between resource efficiency and expression complexity for an FPGA accelerator targeting string-matching operators (LIKE and REGEXP LIKE in SQL).

• “High Throughput Large Scale Sorting on a CPU-FPGA Heterogeneous Platform”

• 2.9x and 1.9x compared with CPU-only and FPGA-only baselines • 2.3x vs. FPGA implementation for sorting

On Going work

• Previous Work• Modulo Scheduling

• Virtual CGRA

• High Level Stream Computation mapped onto HARP

Every day, we create 2.5 quintillion bytes of data —

so much that 90% of the data in the world today

has been created in the last two years alone.

IBM Big-Data

End loop

streams

Sequential

CodeParallel

Data Flow

streams

FU FU FU

Physical

Parallel

Architecture

RUNTIME

Loop Unrolling - Modulo Scheduling

streams

Iteration i

Iteration i+1

Iteration i+2

Iteration i+3

Overlap iterations !

Iteration i

Iteration i+1

Iteration i+2

Iteration i+3

At same time, all operations are executed.....

One Clock Cycle THROUGHPUT !

ILP=10

Iteration i

Iteration i+1

Iteration i+2

Iteration i+3

Physical

Architecture

Iteration i

Iteration i+1

Iteration i+2

Iteration i+3

Physical

Architecture

Iteration i

Iteration i+1

Iteration i+2

Iteration i+3

Physical

Architecture

Iteration i

Iteration i+1

Iteration i+2

Iteration i+3

Physical

Architecture

10 OPs

6 units

6 Units, t3

6 Units, t4

6 Units, t5

New result, every 2 cycles ….ILP=5

Initial Interval (II) = 2 cycles 6 Units, t6

Configuration C0

placement

Configuration C0

Configuration C1

Placement and

Routing

Configuration C0

Configuration C1

Placement and

Routing

Iteration i

Iteration i+1

Configuration C0

Configuration C1

Placement and

Routing

Iteration i

Iteration i+1

Configuration C1

Configuration C0

Configuration Memory

Physical

Architecture

Iteration i

Iteration i+1

Configuration C1

Configuration C0

Configuration Memory

Physical

Architecture

Global Register

FUFUFUFU

RF RF RF RF

Virtual CGRA on the top of

Commercial FPGA

XILINX XC6VLX75T

FlipFlop 2.5 %

LUTs 14.7 %

Mem Bank 16.0 %

Clock 110 Mhz

FlipFlop 2.7 %

LUTs 17.6 %

Mem Bank 4.5 %

Clock 90 Mhz

Virtualization

Questions ?

ricardo@ufv.br

jnacif@ufv.br

VI Escola de Sistemas Embarcados - INFcaco/ESSE2016/ESSE2016_Nacif.pdf · VI Escola de Sistemas...

Documents

Transcript of VI Escola de Sistemas Embarcados - INFcaco/ESSE2016/ESSE2016_Nacif.pdf · VI Escola de Sistemas...

Escola Secundáriaa - ULisboarepositorio.ul.pt/bitstream/10451/7301/10/ulsd063820_td_anexo9.pdf · Artigo 3º - Oferta educativa ... Anexo VI – Estatutos da Associação de Estudantes

Revista de Prensa - girovi.cat · 14 Dominical Diumenge 2 de setembre de 2012 Agustí Ensesa Bonet Escola de Tastavins del Gironès Diari de ... un vi de més estructura El vi MO

ESCOLA BILÍNGUE vs . ESCOLA INTERNACIONAL: CONCEITOS, …repositorio.roca.utfpr.edu.br/jspui/bitstream/1/13600/1/... · 2019. 12. 9. · ESCOLA BILÍNGUE vs . ESCOLA INTERNACIONAL:

DELS CENTRES EDUCATIUS DE L’HOSPITALET … · 5. 57 pablo neruda, escola. 44 paco candel, escola. 56 patufet sant jordi, escola. 69 pau casals, escola. 53 pau sans, escola. 70 pau

LA LLIGA DELS LLIBRES 2012 - 2013 - l-h.cat · 12. ESCOLA LA MARINA 13. ESCOLA MILAGROS CONSARNAU – A 14. ESCOLA MILAGROS CONSARNAU – B 15. ESCOLA PABLO NERUDA 16. ESCOLA PATUFET

DELS CENTRES EDUCATIUS · 5 50 patufet sant jordi, escola 63 pau casals, escola 47 pau sans, escola 64 pau vila, escola 56 pep ventura, escola 62 pere lliscart, escola 36 pompeu fabra,

2015 04 26 escola 12 escola

3M Marina, embarcados contigo en cualquier reparación ...pinmarsupply.com/media/wp-content/uploads/2015/03/Folletos-Marina-2016.pdf3M Marina, embarcados contigo en cualquier reparación

XARXA DE Cb (primària) · Escola Benviure Escola Font freda Escola President Salvans Escola Schola ... Escola Sant Martí • Escola El turó • • Escola Saltells . ROSER CANALS

VI MOSTRA CINEMA ESTC - Amadora...nas áreas das Artes Performativas de Teatro e de Cinema com origem na primeira Escola de Teatro portuguesa, a Escola de Arte Dramática, depois Escola

traición… donde aguarda el destino. Embarcados en un viaje

Sistemas LIDAR embarcados en drones TerraSystem · PDF fileCongreso sobre las Aplicaciones de los DRONES a la Ingeniería Civil 24 - 25 enero 2017 Sistemas LIDAR embarcados en drones

Protocolos de Sistemas Embarcados

...Escola Bressol Nova Fortuny Escola Charlie Rive' Escola Ernest Lluch Escola Infantil Pinocho Escola Màrius Torres Escola Pep Ventura Escola SantJaume de la Fep Escola Santiago

2014 06 07 escola 12 escola

Sistemas Embarcadoscm-kls-content.s3.amazonaws.com/201801/INTERATIVAS_2_0/...embarcados e, finalmente, como é feita uma configuração de redes para sistemas embarcados. Na Unidade

Consorci d'Educació de Barcelona · 2016-10-14 · Escola Doctor Ferran i Clua Escola Dolors Monserdà-Santapau Escola ... Escola Ferran Sunyer Escola Ferrer i Guàrdia Escola Fluvià

ESCOLES - El Pregó Dels Infants · CURS 2012-2013: les aules i per la xarxa, en el procés Escola Aiguamarina – Escola Barrufet Escola Gavina – Escola Paideia Escola Projecte

Ambiente de simulação para sistemas embarcados fileALAN FISCHER E SILVA DANILO CESAR LEMES DE PAULA Ambiente de simulação para sistemas embarcados Trabalho de Graduaçao em

ANÁLISIS DEL CONTRATO DE LOS TRABAJADORES EMBARCADOS O GENTE DE MAR Y LEGISLACIÓN