redpitaya-puzzlefw/doc/fpga_firmware.md

PuzzleFW FPGA firmware

The PuzzleFW firmware provides the following functionality:

• Collect ADC samples at 125 MSa/s with configurable decimation or averaging.
• Trigger on external digital input and collect a configurable number of samples.
• Detect transitions on digital input signals and assign time tags to these events.
• Transfer ADC data and timetagger data to the Zynq PS via DMA.

Status LEDs

LED0 to LED7 are the first 8 yellow LEDs from left to right on the side of the Red Pitaya.

LED0 blinks at a rate of 1 Hz when the PuzzleFW firmware is active. Its purpose is to provide a minimal indication that the FPGA is active.

LED1 is on when the analog acquisition chain is enabled (register ACQUISITION_EN).

LED2 is on when the analog acquisition chain is waiting for a trigger.

LED3 is on when the timetagger is active (at least one event type is enabled).

LED4 to LED7 are controlled by software via register LED_STATE.

GPIO signals

The GPIO controller in the Zynq PS is internally connected to the FPGA. A few of the internal GPIO signals are used by the firmware. By manipulating these GPIO signals, the Linux system can reset the FPGA firmware and control specific features.

EMIO GPIO pin	Linux GPIO	Direction	Description
0	54	out (to FPGA)	Global firmware reset
1	55	in (from FPGA)	Reset status report
2	56	out (to FPGA)	ADC clock duty cycle stabilizer

GPIO 0: Global firmware reset

GPIO 0 drives a global reset signal to the FPGA firmware.

• GPIO 0 is an output from the PS to the FPGA.
• Driving this signal as 0 resets the FPGA firmware.
• Driving this signal as 1 releases the firmware reset.

This signal should be used to reset the FPGA during or after any manipulations of the ADC clock signal such as switching the ADC clock duty cycle stabilizer.

After switching this signal to 1, it may take approximately 1 ms for the FPGA to lock clocks and release its reset.

Note: Any attempt to access firmware registers via the AXI bus while the FPGA is in reset, will crash the Zynq PS. For that reason, the output state of GPIO 0 must remain at 1 as long as any software is interacting with the FPGA.

GPIO 1: Reset status report

GPIO 1 reports the reset status of the FPGA firmware.

• GPIO 1 is an input to the PS from the FPGA.
• The signal is 0 while the FPGA is in reset.
• The signal is 1 if the FPGA reset has been released.

GPIO 2: ADC clock duty cycle stabilizer

GPIO 2 drives the clock duty cycle stabilizer signal of the LTC2145 ADC.

• GPIO 2 is an output from the PS to the FPGA.
• Driving this signal as 0 disables the clock duty cycle stabilizer.
• Driving this signal as 1 enables the clock duty cycle stabilizer.

This signal is only supported on Red Pitaya boards with 2 input channels.

This GPIO signal is asynchronously routed through the FPGA to the ADC. Changing the state of this signal may shift the phase of the ADC data clock. It must therefore only be changed while the FPGA firmware is in reset.

SPI signals

The SPI controller SPI0 in the Zynq PS is internally connected to the FPGA. The SPI signals from this controller are asynchronously routed through the FPGA to the LTC2145 ADCs.

This feature is only supported on Red Pitaya boards with 4 input channels. Boards with 2 input channels do not provide access to the SPI bus of the ADC.

The SPI signals are connected as follows:

SPI signal	Direction	Connection
SCLK	out	SCK pin of both ADCs
MOSI	out	SDI pin of both ADCs
MISO	in	wired to 0
SS[0]	out	CSn pin of ADC A
SS[1]	out	CSn pin of ADC B
SS[2]	out	not connected

Note that programming the timing registers of the ADC, may shift the phase of the ADC data clock. It must therefore only be done while the FPGA firmware is in reset.

Analog acquisition chain

The analog acquisition chain has 2 analog input signals (or 4 channels for a 4-channel device). The analog signals are simultaneously sampled with ADCs at a fixed sample rate of 125 MSa/s. Raw ADC samples are 14-bit unsigned integers.

The FPGA can optionally reduce the sample rate through decimation or averaging. When a trigger condition occurs, the FPGA collects a sequence of decimated samples and transfers them via DMA. Continuous streaming acquisition is supported via an auto-trigger mode that triggers immediately when a previous acquisition ends.

Sample data are queued in a FIFO buffer inside the FPGA before being transferred via DMA. This FIFO is necessary to hold data for a short time while the DMA engine sets up a DMA transfer. It can also be used to collect a limited number of samples at the maximum sample rate without decimation, which produces a higher data rate than can be sustained by DMA. The FIFO has room for 16384 messages.

Averaging and decimation

The ADCs operate at a fixed sample rate of 125 MSa/s. While the sample rate of the ADC can not be changed, the effective sample rate can be reduced by digital processing in the FPGA.

The effective sample rate after digital processing is equal to the ADC sample rate divided by the decimation factor. The decimation factor is an integer, configured via register DECIMATION_FACTOR, which contains the decimation factor minus 1. Setting this register to 0 configures decimation factor 1 such that the effective rate is identical to the ADC sample rate. Setting a higher decimation factor reduces the effective sample rate accordingly.

The FPGA supports two modes of sample rate reduction: decimation and averaging. In decimation mode with decimation factor N, the FPGA keeps one ADC sample, then discards N - 1 samples, then keeps one ADC sample, discards N - 1 sample and so on. In averaging mode, the FPGA calculates the sum of each group of N samples.

Decimation mode has the significant disadvantage that it folds aliased high frequency signals into the downsampled signal. Averaging mode has the advantage that it suppresses aliasing while also reducing the impact of quantization noise from the ADC. For these reasons, averaging mode is the best choice in most cases.

If averaging mode is used with a high decimation factor (1024 or higher), the sum of a group of ADC samples may not fit in a 24-bit sample word. In this case, the summed value may be shifted right by a configurable number of bit positions, effectively dividing the sum by a power of 2. Before shifting right, a bias value is added such that the result represents the sum divided by a power of 2 and rounded to the nearest integer, or rounded up in case of a tie.

We use the term decimated samples to refer to samples that are produced by this sample rate reduction process, even when averaging mode is active, and even when the effective sample rate is equal to the ADC sample rate.

Triggered acquisition

When a trigger occurs, the acquisition chain collects a configurable number of decimated samples. The number of samples per trigger is configured via register RECORD_LENGTH.

When auto-trigger mode is active, the acquisition chain triggers continuously. A new trigger occurs as soon as acquisition for the previous trigger has ended, after a dead time controlled by TRIGGER_DELAY. In this mode, the decimation factor must be at least 2 (or at least 4 in 4-channel mode). If TRIGGER_DELAY is zero, sampling continues accross acquisitions at a fixed pace controlled by the decimation factor. This makes it possible to set up continuous streaming sampling.

When external triggering is active, sampling starts when the specified external trigger condition occurs, after a delay of TRIGGER_DELAY samples. The external trigger event is subject to a jitter of 1 sample (8 ns). Further trigger events are ignored until acquisition for the previous trigger has ended.

When auto-trigger mode and external trigger mode are both disabled, the acquisition chain remains idle. In this mode, a trigger event can be forced by an explicit action from software.

Output data format

The output from the analog acquisition chain is a sequence of 64-bit messages. Three types of messages are used:

• Sample messages, containing a pair of samples from two channels.
• Trigger messages, emitted when a triggered acquisition starts.
• Overflow messages, emitted when the internal data buffer overflows.

Acquisition sample message

Bits	Field name	Description
63 : 56	msg_type	Fixed value 0x10
55 : 52	channel1	Value 0x1 or 0x3 to indicate channel IN2 or IN4.
51 : 48	channel0	Value 0x0 or 0x2 to indicate channel IN1 or IN3.
47 : 24	sample1	Sample value from channel IN2 or IN4.
23 : 0	sample0	Sample value from channel IN1 or IN3.

In 2-channel mode, a sample message is emitted for each decimated sample produced during a triggered acquisition. Each sample message contains a pair of samples from channels IN1 and IN2.

In 4-channel mode (only supported for 4-channel devices), a pair of sample messages is emitted for each decimated sample. The first message of each pair contains samples from channels IN1 and IN2. The second message of each pair contains samples from channels IN3 and IN4.

Samples are unsigned 24-bit integers. In decimation mode (averaging off), the value represents the ADC code of the first sample of the decimation group. In averaging mode, the value represents the sum of all ADC codes in the decimation group.

Acquisition trigger message

Bits	Field name	Description
63 : 56	msg_type	Fixed value 0x11
47 : 0	timestamp	Timestamp in units of 8 ns

A trigger message is emitted when a triggered acquisition starts. The trigger message is emitted before the first sample of the acquisition.

The trigger timestamp represents the moment when integration of the first sample starts.

Acquisition overflow message

Bits	Field name	Description
63 : 56	msg_type	Fixed value 0x40

An overflow message is emitted when the internal data buffer overflows. It marks the point in the message sequence where an unknown number of messages have been discarded.

Timetagger

The timetagger has 4 digital input signals. If a rising or falling edge occurs on one of these signals, a timestamp is assigned to that event and a message is emitted and transferred via DMA.

A 4-cycle glitch filter is applied to the digital input signals. This filter rejects digital pulses shorter than 4 clock cycles (32 ns).

Messages are only emitted for enabled event types. Rising edges and falling edges can be separately enabled or disabled for each digital input channel. This is done via register TIMETAGGER_EN.

The messages are queued in a FIFO buffer inside the FPGA before being transferred via DMA. This FIFO is necessary to hold messages for a short time while the DMA engine sets up a DMA transfer. It has room for 4096 messages.

Output data format

The output from the timetagger is a sequence of 64-bit messages. Three types of messages are used:

• Event messages, emitted when a digital input event occurs that it enabled for time tagging.
• Marker messages, emitted on explicit request from software.
• Overflow messages, emitted when the internal data buffer overflows.

Timetagger event message

Bits	Field name	Description
63 : 60	msg_type	Fixed value 0x2
59 : 57	channel	Index of digital input channel where the event occurred.
56	falling_edge	'1' if a falling edge occurred, '0' if a rising edge occurred.
51 : 48	input_state	New state of digital input signals after the event.
47 : 0	timestamp	Timestamp in units of 8 ns

An event message is emitted when a digital input event occurs that is enabled for time tagging.

Note that multiple events can occur simultaneously. If this happens, separate messages are emitted for all events that occurred. These messages all have the same timestamp, and the same input_state field.

Timetagger marker messages

Bits	Field name	Description
63 : 56	msg_type	Fixed value 0x30
51 : 48	input_state	State of digital input signals.
47 : 0	timestamp	Timestamp in units of 8 ns

A marker message is emitted when software writes value 1 to register TIMETAGGER_MARK. This may be used to establish synchronization between digital input events and some other type of events that are triggered or detected by software.

Timetagger overflow message

Bits	Field name	Description
63 : 56	msg_type	Fixed value 0x40

An overflow message is emitted when the internal data buffer overflows. It marks the point in the message sequence where an unknown number of messages have been discarded.

Registers

The behaviour of the FPGA is controlled via a set of registers. These registers are accessible from the Zynq PS via memory mapped I/O.

The registers are located in a 2 MB region with address 0x43000000 to 0x431fffff in the PS address map.

Register access is implemented via an internal APB bus inside the FPGA. This APB bus is connected via an intermediate AXI bus to port M_AXI_GP0 of the Zynq PS-to-PL interface.

Within the register address space, the first 1 MB area is intended for various types of control and status registers that are accessed by user space software. The second 1 MB area is intended for privileged registers that are only accessed by system software such as the Linux kernel driver. The difference is that writing incorrect values to privileged registers can corrupt or crash the PS system, while writing to user registers can not.

The complete set of registers is listed below. Access to undefined addresses within the register area must be avoided. If such access occurs, the FPGA may either ignore it or map it to an existing register in a way that may be unexpected.

Register list

This list specifies register addresses as offsets within the register address space. The base address 0x43000000 must be added to obtain the absolute address on the PS memory bus.

User registers

Address	Name	Description
0x000000	INFO	Firmware version information
0x000010	IRQ_ENABLE	Enable PL-to-PS interrupts
0x000014	IRQ_PENDING	Mask of pending interrupt conditions
0x000100	DMA_EN	Enable DMA transfers
0x000104	DMA_STATUS	Status of DMA engine
0x000108	DMA_CLEAR	Clear DMA error flags
0x000180	TIMESTAMP_LO	Low 32 bits of global timestamp
0x000184	TIMESTAMP_HI	High 16 bits of global timestamp
0x000188	TIMESTAMP_CLEAR	Reset timestamp counter to zero
0x000200	ACQ_ADDR_START	Start address of DMA buffer for analog acquisition
0x000204	ACQ_ADDR_END	End address of DMA buffer for analog acquisition
0x000208	ACQ_ADDR_LIMIT	Pause DMA when write pointer reaches this point
0x00020C	ACQ_ADDR_INTR	Raise interrupt when write pointer reaches thi point
0x000210	ACQ_ADDR_PTR	Current DMA write pointer
0x000214	ACQ_DMA_CTRL	Control DMA write stream for analog acquisition
0x000218	ACQ_INTR_CTRL	Control DMA interrupts for analog acquisition
0x00021C	ACQ_DMA_STATUS	Status of DMA write stream for analog acquisition
0x000220	ACQUISITION_EN	Enable analog data acquisition
0x000224	RECORD_LENGTH	Number of samples per trigger minus 1.
0x000228	DECIMATION_FACTOR	Sample decimation factor minus 1.
0x00022C	SHIFT_STEPS	Nr of bit steps to shift accumulated value right
0x000230	AVERAGING_EN	Select averaging or decimation
0x000234	CH4_MODE	Enable 4-channel sampling (4-channel device only)
0x000238	SIMULATE_ADC	Simulate ADC samples
0x000240	TRIGGER_MODE	Trigger configuration
0x000244	TRIGGER_DELAY	Trigger delay
0x000248	TRIGGER_STATUS	Trigger status
0x000280	ADC_SAMPLE	Most recent ADC samples from IN1 and IN2
0x000284	ADC23_SAMPLE	Most recent ADC samples from IN3 and IN4 (4-channel device only)
0x00028C	ADC_RANGE_CLEAR	Clear min/max ADC sample values
0x000290	ADC0_MINMAX	Min/max ADC sample values from IN1
0x000294	ADC1_MINMAX	Min/max ADC sample values from IN2
0x000298	ADC2_MINMAX	Min/max ADC sample values from IN3 (4-channel device only)
0x00029C	ADC3_MINMAX	Min/max ADC sample values from IN4 (4-channel device only)
0x000300	TT_ADDR_START	Start address of DMA buffer for timetagger
0x000304	TT_ADDR_END	End address of DMA buffer for timetagger
0x000308	TT_ADDR_LIMIT	Pause DMA when write pointer reaches this point
0x00030C	TT_ADDR_INTR	Raise interrupt when write pointer reaches thi point
0x000310	TT_ADDR_PTR	Current DMA write pointer
0x000314	TT_DMA_CTRL	Control DMA write stream for timetagger
0x000318	TT_INTR_CTRL	Control DMA interrupts for timetagger
0x00031C	TT_DMA_STATUS	Status of DMA write stream for timetagger
0x000320	TIMETAGGER_EN	Mask of enabled timetagger channels
0x000324	TIMETAGGER_MARK	Emit marker record in timetagger data
0x000330	DIG_SIMULATE	Simulate digital input
0x000338	DIG_SAMPLE	Current state of digital input signals
0x000404	LED_STATE	Control LEDs

Privileged registers

Address	Name	Description
0x100000	DMA_BUF_ADDR	Base address of DMA buffer in RAM
0x100004	DMA_BUF_SIZE	Size of DMA buffer in RAM

Register descriptions

Registers are 32 bits wide, but many registers use only a subset of the bits. A field is a range of meaningful bits within a register. Some registers contain multiple fields.

Fields are marked as RW or RO or WC.

• RW fields can be modified by writing to the register. Reading from the register returns the most recent value written to the field.
• RO fields are read-only. Writes to these fields are ignored. The value of these fields typically changes spontaneously to reflect the status of the FPGA.
• WC fields are write-only and auto-clearing. Writing a 1 bit to the field triggers an action in the FPGA. Writing a 0 bit has no effect. WC fields are returned as 0 when reading from the register.

Bits that are not part of a field are reserved. Reserved bits must be ignored when reading. Reserved bits should be written as zero or left unmodified when writing.

0x000000: INFO

Firmware version information.

Bits	Field name	Access	Description
7 : 0	version_minor	RO	Firmware minor version
15 : 8	version_major	RO	Firmware major version
23 : 16	api_version	RO	Firmware API version, currently 0x01
31 : 24		RO	Fixed value 0x4a

0x000010: IRQ_ENABLE

Enable PL-to-PS interrupts.

Bits	Field name	Access	Description
0	irq_enable	RW	'1' to enable interrupts, '0' to disable

This register acts as a master switch for interrupts from the FPGA. Specific interrupt conditions can be separately enabled or disabled via other registers.

0x000014: IRQ_PENDING

Mask of pending interrupt conditions.

Bits	Field name	Access	Description
0	acq_intr	RO	'1' if an interrupt is pending for analog acquisition DMA
1	tt_intr	RO	'1' if an interrupt is pending for timetagger DMA

0x000100: DMA_EN

Enable DMA transfers.

Bits	Field name	Access	Description
0	dma_en	RW	'1' to enable DMA transfers, '0' to disable

This register acts as a master switch for the DMA engine in the FPGA.

Disabling DMA will prevent new bursts from starting, but allows ongoing bursts to complete. Under normal circumstances, the completion of ongoing bursts can take at most a few microseconds.

0x000104: DMA_STATUS

Status of DMA engine.

Bits	Field name	Access	Description
0	dma_busy	RO	'1' if a DMA burst is in progres
1	err_read	RO	'1' if a DMA read access failed
2	err_write	RO	'1' if a DMA write access failed
3	err_address	RO	'1' if access outside the DMA buffer was attempted
4	err_any	RO	'1' if any DMA error occurred

If a DMA error occurs, no further DMA bursts will be started until the error is explicitly cleared. The failed DMA burst will not be acknowledged to the subsystem that initiated it (analog or timetagger). To recover from this situation, it will typically be necessary to stop and re-initialize all ongoing DMA data streams, clear the error, then restart DMA data streams.

An address error typically occurs when the DMA address range for the analog subsystem or the timetagger exceeds the size of the DMA buffer specified via DMA_BUF_SIZE.

0x000108: DMA_CLEAR

Clear DMA error flags.

Bits	Field name	Access	Description
0	dma_clear	WC	Write '1' to clear pending DMA errors

0x000180: TIMESTAMP_LO

Low 32 bits of global timestamp.

Bits	Field name	Access	Description
31 : 0		RO	Least significant 32 bits of the global 8 ns timestamp counter.

0x000184: TIMESTAMP_HI

High 16 bits of global timestamp.

Bits	Field name	Access	Description
15 : 0		RO	Most significant 16 bits of the global 8 ns timestamp counter.

When reading both TIMESTAMP_LO and TIMESTAMP_HI, note that the timestamp will change between the two reads, possibly resulting in an invalid combination of low and high part. To avoid this, read TIMESTAMP_HI, then TIMESTAMP_LO, then TIMESTAMP_HI again. Then check that the value of TIMESTAMP_HI has not changed during the sequence, otherwise retry.

0x000188: TIMESTAMP_CLEAR

Reset timestamp counter to zero.

Bits	Field name	Access	Description
0	timestamp_clear	WC	Write '1' to reset the timestamp counter.

0x000200: ACQ_ADDR_START

Start address of DMA buffer for analog acquisition.

Bits	Field name	Access	Description
31 : 7		RW	Start address as multiple of 128 bytes.

The address must be aligned to a 128-byte boundary. The address is interpreted as an offset relative to DMA_BUF_ADDR.

0x000204: ACQ_ADDR_END

End address of DMA buffer for analog acquisition.

Bits	Field name	Access	Description
31 : 7		RW	End address as multiple of 128 bytes.

The end address points past the end of the DMA buffer. When the DMA write pointer reaches ACQ_ADDR_END, it immediately wraps back to ACQ_ADDR_START.

The address must be aligned to a 128-byte boundary. The address is interpreted as an offset relative to DMA_BUF_ADDR.

0x000208: ACQ_ADDR_LIMIT

Pause DMA when write pointer reaches this point.

Bits	Field name	Access	Description
31 : 7		RW	Address as multiple of 128 bytes.

When the address pointer reaches the limit address, the DMA engine pauses further transfers until software updates the limit.

The address must be aligned to a 128-byte boundary. The address is interpreted as an offset relative to DMA_BUF_ADDR.

0x00020C: ACQ_ADDR_INTR

Raise interrupt when write pointer reaches this point.

Bits	Field name	Access	Description
31 : 3		RW	Address as multiple of 8 bytes.

When the address pointer reaches this address, an interrupt occurs if analog acquisition interrupts are enabled.

The address must be aligned to an 8-byte boundary. The address is interpreted as an offset relative to DMA_BUF_ADDR.

0x000210: ACQ_ADDR_PTR

Current DMA write pointer.

Bits	Field name	Access	Description
31 : 3		RO	Address as multiple of 8 bytes.

This register indicates the address where the next write transfer will start. Write transfers have completed for locations up to (but excluding) this address. This indicates that software can read valid data up to (but excluding) the pointer address.

The address will be aligned to an 8-byte boundary. The address is interpreted as an offset relative to DMA_BUF_ADDR.

0x000214: ACQ_DMA_CTRL

Control DMA write stream for analog acquisition.

Bits	Field name	Access	Description
0	acq_dma_en	RW	'1' to enable DMA transfers for analog acquisition.
1	acq_dma_init	WC	Write '1' to initialize the analog acquisition DMA stream

Initializing the DMA stream resets the write pointer to ACQ_ADDR_START and discards pending data from the internal buffer. Initializing is necessary after setting up the buffer address, and to recover from DMA errors.

0x000218: ACQ_INTR_CTRL

Control DMA interrupts for analog acquisition.

Bits	Field name	Access	Description
0	acq_intr_en	RW	'1' to enable interrupt for DMA write pointer.
1	acq_intr_clear	WC	Write '1' to clear a pending interrupt condition.

0x00021C: ACQ_DMA_STATUS

Status of DMA write stream for analog acquisition.

Bits	Field name	Access	Description
0	acq_dma_busy	RO	'1' if an uncompleted DMA burst is in progress.

0x000220: ACQUISITION_EN

Enable analog data acquisition.

Bits	Field name	Access	Description
0	acquisition_en	RW	'1' to enable analog data acquisition.

When analog acquisition is enabled, triggers are processed according to the configured trigger mode and samples are collected accordingly. If analog acquisition is disabled, any ongoing sample collection stops immediately and further triggers are ignored.

0x000224: RECORD_LENGTH

Number of samples per trigger minus 1.

Bits	Field name	Access	Description
15 : 0	record_length	RW	Number of samples per trigger minus 1.

When a trigger condition occurs, the acquisition chain collects (record_length + 1) decimated samples.

0x000228: DECIMATION_FACTOR

Sample decimation factor minus 1.

Bits	Field name	Access	Description
17 : 0	decimation_factor	RW	Decimation factor minus 1.

The ADCs sample at a fixed sample rate of 125 MSa/s. The resulting sample stream is decimated by a factor (decimation_factor + 1).

If averaging is enabled, the sum of each group of (decimation_factor + 1) ADC samples is collected as a decimated sample. If averaging is disabled, only the first sample of each group of (decimation_factor + 1) ADC samples is collected.

0x00022C: SHIFT_STEPS

Number of bit positions to shift accumulated value right.

Bits	Field name	Access	Description
3 : 0	shift_steps	RW	Number of bit positions.

The accumulated value of each decimated sample is right-shifted by the configured number of bit positions before the value is transferred to DMA. This is necessary to prevent overflow when a high decimation factor (above 1024) is used in averaging mode.

0x000230: AVERAGING_EN

Select averaging or decimation.

Bits	Field name	Access	Description
0	averaging_en	RW	'1' to average samples, '0' to decimate samples.

0x000234: CH4_MODE

Enable 4-channel sampling (only supported on 4-channel devices).

Bits	Field name	Access	Description
0	ch4_mode	RW	'1' to enable 4-channel sampling.
8	ch4_enabled	RO	'1' for a 4-channel device.

When 4-channel sampling is disabled, only channels IN1 and IN2 are active. One 64-bit word is emitted for each decimated sample, containing the data of these two channels.

When 4-channel sampling is enabled, channels IN3 and IN4 are also active. Two 64-bit words are emitted for each decimated sample. The first word contains the data for channels IN1 and IN2. The second word contains the data for channels IN3 and IN4.

When 4-channel sampling is enabled, the decimation factor must be at least 2 (DECIMATION_FACTOR at least 1).

0x000238: SIMULATE_ADC

Simulate ADC samples.

Bits	Field name	Access	Description
0	simulate_adc	RW	'1' to use simulated ADC samples, '0' to use real ADC samples.

0x000240: TRIGGER_MODE

Trigger configuration.

Bits	Field name	Access	Description
0	trig_auto_en	RW	'1' to enable auto-trigger mode.
1	trig_ext_en	RW	'1' to enable external triggering.
2	trig_ext_once	RW	'1' to accept one external trigger.
5 : 4	trig_ext_select	RW	Select digital input channel to use as external trigger.
7	trig_ext_falling	RW	'1' to trigger on falling edge, '0' to trigger on rising edge.
8	trig_force	WC	Write '1' to force a single trigger event.

Auto-trigger mode takes precedence over external triggering. If auto-triggering and external triggering are both disabled, no data acquisition takes place unless a trigger is forced by writing to trig_force.

External triggering is enabled if either trig_ext_en or trig_ext_once is set. If any trigger occurs (regardless of trigger source), trig_ext_once is cleared automatically.

0x000244: TRIGGER_DELAY

Trigger delay.

Bits	Field name	Access	Description
15 : 0	trigger_delay	RW	Trigger delay in samples (8 ns).

0x000248: TRIGGER_STATUS

Trigger delay.

Bits	Field name	Access	Description
0	trig_waiting	RO	'1' if the acquisition chain is waiting for a trigger.

0x000280: ADC_SAMPLE

Most recent ADC samples from IN1 and IN2.

Bits	Field name	Access	Description
13 : 0	adc0_sample	RO	ADC sample for channel IN1.
29 : 16	adc1_sample	RO	ADC sample for channel IN2.

0x000284: ADC_SAMPLE

Most recent ADC samples from IN3 and IN4 (only supported on 4-channel devices).

Bits	Field name	Access	Description
13 : 0	adc2_sample	RO	ADC sample for channel IN3.
29 : 16	adc3_sample	RO	ADC sample for channel IN4.

0x00028C: ADC_RANGE_CLEAR

Most recent ADC samples from IN3 and IN4 (only supported on 4-channel devices).

Bits	Field name	Access	Description
0	adc_range_clear	WC	Write '1' to clear min/max ADC sample values.

0x000290: ADC0_MINMAX

Minimum and maximum ADC sample from IN1.

Bits	Field name	Access	Description
13 : 0	adc0_min	RO	Minimum ADC sample from IN1 since range cleared.
29 : 16	adc0_max	RO	Maximum ADC sample from IN1 since range cleared.

0x000294: ADC1_MINMAX

Minimum and maximum ADC sample from IN2.

Bits	Field name	Access	Description
13 : 0	adc1_min	RO	Minimum ADC sample from IN2 since range cleared.
29 : 16	adc1_max	RO	Maximum ADC sample from IN2 since range cleared.

0x000298: ADC2_MINMAX

Minimum and maximum ADC sample from IN3 (only supported on 4-channel devices).

Bits	Field name	Access	Description
13 : 0	adc2_min	RO	Minimum ADC sample from IN3 since range cleared.
29 : 16	adc2_max	RO	Maximum ADC sample from IN3 since range cleared.

0x00029C: ADC3_MINMAX

Minimum and maximum ADC sample from IN4 (only supported on 4-channel devices).

Bits	Field name	Access	Description
13 : 0	adc3_min	RO	Minimum ADC sample from IN4 since range cleared.
29 : 16	adc3_max	RO	Maximum ADC sample from IN4 since range cleared.

0x000300: TT_ADDR_START

Start address of DMA buffer for timetagger.

Bits	Field name	Access	Description
31 : 7		RW	Start address as multiple of 128 bytes.

The address must be aligned to a 128-byte boundary. The address is interpreted as an offset relative to DMA_BUF_ADDR.

0x000304: TT_ADDR_END

End address of DMA buffer for timetagger.

Bits	Field name	Access	Description
31 : 7		RW	End address as multiple of 128 bytes.

The end address points past the end of the DMA buffer. When the DMA write pointer reaches TT_ADDR_END, it immediately wraps back to TT_ADDR_START.

The address must be aligned to a 128-byte boundary. The address is interpreted as an offset relative to DMA_BUF_ADDR.

0x000308: TT_ADDR_LIMIT

Pause DMA when write pointer reaches this point.

Bits	Field name	Access	Description
31 : 7		RW	Address as multiple of 128 bytes.

When the address pointer reaches the limit address, the DMA engine pauses further transfers until software updates the limit.

The address must be aligned to a 128-byte boundary. The address is interpreted as an offset relative to DMA_BUF_ADDR.

0x00030C: TT_ADDR_INTR

Raise interrupt when write pointer reaches this point.

Bits	Field name	Access	Description
31 : 3		RW	Address as multiple of 8 bytes.

When the address pointer reaches this address, an interrupt occurs if analog acquisition interrupts are enabled.

The address must be aligned to an 8-byte boundary. The address is interpreted as an offset relative to DMA_BUF_ADDR.

0x000310: TT_ADDR_PTR

Current DMA write pointer.

Bits	Field name	Access	Description
31 : 3		RO	Address as multiple of 8 bytes.

This register indicates the address where the next write transfer will start. Write transfers have completed for locations up to (but excluding) this address. This indicates that software can read valid data up to (but excluding) the pointer address.

The address will be aligned to an 8-byte boundary. The address is interpreted as an offset relative to DMA_BUF_ADDR.

0x000314: TT_DMA_CTRL

Control DMA write stream for timetagger.

Bits	Field name	Access	Description
0	tt_dma_en	RW	'1' to enable DMA transfers timetagger.
1	tt_dma_init	WC	Write '1' to initialize the timetagger DMA stream

Initializing the DMA stream resets the write pointer to TT_ADDR_START and discards pending data from the internal buffer. Initializing is necessary after setting up the buffer address, and to recover from DMA errors.

0x000318: TT_INTR_CTRL

Control DMA interrupts for timetagger.

Bits	Field name	Access	Description
0	tt_intr_en	RW	'1' to enable interrupt for DMA write pointer.
1	tt_intr_clear	WC	Write '1' to clear a pending interrupt condition.

0x00031C: TT_DMA_STATUS

Status of DMA write stream for timetagger.

Bits	Field name	Access	Description
0	tt_dma_busy	RO	'1' if an uncompleted DMA burst is in progress.

0x000320: TIMETAGGER_EN

Mask of enabled timetagger channels.

Bits	Field name	Access	Description
0		RW	'1' to enable rising edge events on digital input 0.
1		RW	'1' to enable falling edge events on digital input 0.
2		RW	'1' to enable rising edge events on digital input 1.
3		RW	'1' to enable falling edge events on digital input 1.
...		RW	...
7		RW	'1' to enable falling edge events on digital input 3.

0x000324: TIMETAGGER_MARK

Emit marker record in timetagger data.

Bits	Field name	Access	Description
0		WC	Write '1' to emit a marker record in the timetagger data stream.

0x000330: DIG_SIMULATE

Simulate digital input.

Bits	Field name	Access	Description
3 : 0	dig_sim_state	RW	Simulated digital input state.
8	dig_sim_en	RW	'1' to use simulated digital input, '0' to use real input.

0x000338: DIG_SAMPLE

Current state of digital input signals.

Bits	Field name	Access	Description
3 : 0	dig_sample	RO	Digital input state.

0x000404: LED_STATE

Control LEDs.

Bits	Field name	Access	Description
7 : 4		RW	Mask controlling 4 yellow LEDs on the side of the Red Pitaya.

0x100000: DMA_BUF_ADDR

Base address of DMA buffer in RAM.

Bits	Field name	Access	Description
31 : 12		RW	Base address of the DMA buffer in physical RAM.

This register controls which region of PS RAM may be accessed via DMA. Typically, the Linux kernel driver must allocate a suitable range of physical memory and initialize these registers accordingly.

The base address must be aligned to a 4 kB boundary.

0x100004: DMA_BUF_SIZE

Size of DMA buffer in RAM.

Bits	Field name	Access	Description
31 : 12		RW	DMA buffer size.

If this register is set to zero, no DMA access is allowed.

DMA

The FPGA acts as an AXI bus master to transfer bulk data to system RAM. This data transfer pattern is commonly referred to as DMA. Note that the Zynq DMA controller (DMAC) is not involved in these data transfers in any way.

DMA access is implemented via an internal AXI bus master inside the FPGA, connected to port S_AXI_HP0 of the Zynq PS-to-PL interface.

Data transfer pattern

The AXI HP data bus is 64 bits wide. All DMA transfers use the full width of the bus on every beat. Therefore, the smallest unit of data transferred via DMA is a 64-bit word. The size of every DMA transfer is a multiple of 8 bytes, and the address of every transfer is aligned to a multiple of 8 bytes.

(This is an implementation choice in the firmware. It is possible in principle to transfer smaller amounts of data via the AXI bus, but the PuzzleFW firmware is designed to transfer 64-bit words in all cases.)

Data are temporarily queued in a FIFO RAM block inside the FPGA until the DMA engine is ready to start a transfer. This is necessary because DMA operates in bursts, and it may take some time before the DMA engine can initiate a burst. The two DMA data streams (analog acquisition, timetagger) each have a separate FIFO memory.

If sufficient data are queued up inside the FPGA, the DMA engine transfers 16-beat bursts (blocks of 128 bytes). This is the most efficient way to use the AXI bus. If the amount of available data is insufficient for a full burst, the DMA engine waits briefly (2 microseconds) for more data to arrive. After waiting, if there are still insufficient data for a full burst, the available data are transferred via single-beat bursts (blocks of 8 bytes).

The highest sustainable data transfer rate is approximately 105,000,000 64-bit words per second, or 800 MiB per second.

Global DMA buffer

A global address window determines the range of allowed target addresses for DMA transfers. This window is configured through the pair of registers DMA_BUF_ADDR and DMA_BUF_SIZE.

Whenever the FPGA initiates a DMA transfer, it first checks that the target address is within the configured buffer. If the transfer falls outside the buffer, it fails and triggers an address error.

The start and size of the DMA buffer must be aligned to 4 kB page boundaries.

Per-stream buffer segments

For each of the two DMA data streams (analog acquisition, timetagger) a dedicated segment may be allocated within the global DMA buffer. These segments are used as circular buffers.

All addresses that specify the start or end of a buffer segment, or any location within a buffer, are specified as offsets relative to the start of the global DMA buffer DMA_BUF_ADDR.

For the analog aqcuisition data stream, the start and end of the buffer segment are configured through registers ACQ_ADDR_START and ACQ_ADDR_END, with ACQ_ADDR_END pointing to the first byte past the end of the segment. The start and end addresses must be aligned to 128-byte boundaries.

The write pointer of the DMA stream is stored in register ACQ_ADDR_PTR. It indicates the destination address of the next DMA transfer. It also signals to software that write transfers have completed for the region up to (but excluding) the write pointer. The write pointer is read-only for software. The FPGA automatically updates the pointer after each data transfer. If the write pointer reaches the end of the segment, it immediately wraps back to the start. The write pointer is always aligned to an 8-byte boundary.

If the write pointer reaches the address configured in ACQ_ADDR_LIMIT, no further DMA transfers will occur until the limit is updated. Software can use this limit to prevent the DMA engine from overwriting parts of the buffer that have not yet been procecessed. The limit address must be aligned to a 128-byte boundary.

If the write pointer reaches the address configured in ACQ_ADDR_INTR, an interrupt is triggered. This happens only if the corresponding interrupt is enabled in registers IRQ_ENABLE and ACQ_INTR_CTRL. The interrupt address must be aligned to an 8-byte boundary.

The timetagger data stream works in the same way as the analog acquisition data stream, but uses a separate set of registers.

Cache coherence

Keep in mind that the Zynq architecture does not provide transparent cache coherence. When the FPGA writes data to system RAM via DMA, the cache of the ARM processor may still contain obsolete data for the same address range.

Software running on the Zynq PS that wants to read from the DMA buffer, must either take explicit steps to invalidate the cache, or it must configure the DMA buffer as non-cacheable memory.

Interrupts

The FPGA can send interrupts to the Zynq PS to notify the software of specific conditions. Interrupts are routed via IRQ_F2P[0], which corresponds to interrupt 61 in the Zynq Generic Interrupt Controller (GIC).

Two types of conditions can trigger an interrupt:

• DMA transfers for analog acquisition have progressed to a configurable point.
• DMA transfers for the timetagger have progressed to a configurable point.

Each type of condition has a separate interrupt enable switch, configured via registers ACQ_INTR_CTRL and TT_INTR_CTRL. Furthermore, there is a master interrupt enable switch, configured via register IRQ_ENABLE.

For example, interrupts may be used to wait for a certain amount of data from the analog acquisition system as follows:

• Write register ACQ_ADDR_INTR to specify the amount of progress after which it wants to be notified.
• Write register ACQ_INTR_CTRL to clear any pending interrupt and enable interrupts on DMA progress.
• Read register ACQ_ADDR_PTR to check whether DMA progress has already passed the configured threshold. This is necessary to avoid a race condition where software keeps waiting for a condition thas has already happened.
• Write register IRQ_ENABLE to enable interrupts.
• Wait for the interrupt or do other stuff in the mean time.
• When the interrupt occurs, the interrupt handler may write register IRQ_ENABLE to disable interrupts until the software has time to handle the cause of the interrupt.
• Optionally read register IRQ_PENDING to confirm that the interrupt was caused by analog data acquisition.
• Write register ACQ_INTR_CTRL to clear the interrupt and disable further interrupts of this type.
• Process the data that arrived in the DMA buffer.

Interrupts can be used to wait for timetagger data in a similar way.