# PuzzleFW FPGA firmware

The PuzzleFW firmware provides the following functionality:

* Collect ADC samples at 125 MSa/s with configurable decimation or averaging.
* Trigger on external digital input and collect a configurable number of samples.
* Detect transitions on digital input signals and assign time tags to these events.
* Transfer ADC data and timetagger data to the Zynq PS via DMA.


# Status LEDs

LED0 to LED7 are the first 8 yellow LEDs from left to right on the side of the Red Pitaya.

LED0 blinks at a rate of 1 Hz when the PuzzleFW firmware is active.
Its purpose is to provide a minimal indication that the FPGA is active.

LED1 is on when the analog acquisition chain is enabled (register `ACQUISITION_EN`).

LED2 is on when the analog acquisition chain is waiting for a trigger.

LED3 is on when the timetagger is active (at least one event type is enabled).

LED4 to LED7 are controlled by software via register `LED_STATE`.

# Analog acquisition chain

The analog acquisition chain has 2 analog input signals (or 4 channels for a 4-channel device).
The analog signals are simultaneously sampled with ADCs at a fixed sample rate of 125 MSa/s.
Raw ADC samples are 14-bit unsigned integers.

The FPGA can optionally reduce the sample rate through decimation or averaging.
When a trigger condition occurs, the FPGA collects a sequence of decimated samples and transfers them via DMA.
Continuous streaming acquisition is supported via an auto-trigger mode that triggers immediately when a previous acquisition ends.

Sample data are queued in a FIFO buffer inside the FPGA before being transferred via DMA.
This FIFO is necessary to hold data for a short time while the DMA engine sets up a DMA transfer.
It can also be used to collect a limited number of samples at the maximum sample rate without decimation, which produces a higher data rate than can be sustained by DMA.
The FIFO has room for 16384 messages.

## Averaging and decimation

The ADCs operate at a fixed sample rate of 125 MSa/s.
While the sample rate of the ADC can not be changed, the effective sample rate can be reduced by digital processing in the FPGA.

The effective sample rate after digital processing is equal to the ADC sample rate divided by the _decimation factor_.
The decimation factor is an integer, configured via register `DECIMATION_FACTOR`, which contains the decimation factor minus 1.
Setting this register to 0 configures decimation factor 1 such that the effective rate is identical to the ADC sample rate.
Setting a higher decimation factor reduces the effective sample rate accordingly.

The FPGA supports two modes of sample rate reduction: decimation and averaging.
In decimation mode with decimation factor _N_, the FPGA keeps one ADC sample, then discards _N_ - 1 samples, then keeps one ADC sample, discards _N_ - 1 sample and so on.
In averaging mode, the FPGA calculates the sum of each group of _N_ samples.

Decimation mode has the significant disadvantage that it folds aliased high frequency signals into the downsampled signal.
Averaging mode has the advantage that it suppresses aliasing while also reducing the impact of quantization noise from the ADC.
For these reasons, averaging mode is the best choice in most cases.

If averaging mode is used with a high decimation factor (1024 or higher), the sum of a group of ADC samples may not fit in a 24-bit sample word.
In this case, the summed value may be shifted right by a configurable number of bit positions, effectively dividing the sum by a power of 2.
Before shifting right, a bias value is added such that the result represents the sum divided by a power of 2 and rounded to the nearest integer, or rounded up in case of a tie.

We use the term _decimated samples_ to refer to samples that are produced by this sample rate reduction process, even when averaging mode is active, and even when the effective sample rate is equal to the ADC sample rate.

## Triggered acquisition

When a trigger occurs, the acquisition chain collects a configurable number of decimated samples.
The number of samples per trigger is configured via register `RECORD_LENGTH`.

When auto-trigger mode is active, the acquisition chain triggers continuously.
A new trigger occurs as soon as acquisition for the previous trigger has ended, after a dead time controlled by `TRIGGER_DELAY`.
In this mode, the decimation factor must be at least 2 (or at least 4 in 4-channel mode).
If `TRIGGER_DELAY` is zero, sampling continues accross acquisitions at a fixed pace controlled by the decimation factor.
This makes it possible to set up continuous streaming sampling.

When external triggering is active, sampling starts when the specified external trigger condition occurs, after a delay of `TRIGGER_DELAY` samples.
The external trigger event is subject to a jitter of 1 sample (8 ns).
Further trigger events are ignored until acquisition for the previous trigger has ended.

When auto-trigger mode and external trigger mode are both disabled, the acquisition chain remains idle.
In this mode, a trigger event can be forced by an explicit action from software.

## Output data format

The output from the analog acquisition chain is a sequence of 64-bit messages.
Three types of messages are used:

* Sample messages, containing a pair of samples from two channels.
* Trigger messages, emitted when a triggered acquisition starts.
* Overflow messages, emitted when the internal data buffer overflows.

### Acquisition sample message

| Bits    | Field name    | Description |
|---------|---------------|-------------|
| 63 : 56 | msg_type      | Fixed value 0x10 |
| 55 : 52 | channel1      | Value 0x1 or 0x3 to indicate channel IN2 or IN4. |
| 51 : 48 | channel0      | Value 0x0 or 0x2 to indicate channel IN1 or IN3. |
| 47 : 24 | sample1       | Sample value from channel IN2 or IN4. |
| 23 : 0  | sample0       | Sample value from channel IN1 or IN3. |

In 2-channel mode, a sample message is emitted for each decimated sample produced during a triggered acquisition.
Each sample message contains a pair of samples from channels IN1 and IN2.

In 4-channel mode (only supported for 4-channel devices), a pair of sample messages is emitted for each decimated sample.
The first message of each pair contains samples from channels IN1 and IN2.
The second message of each pair contains samples from channels IN3 and IN4.

Samples are unsigned 24-bit integers.
In decimation mode (averaging off), the value represents the ADC code of the first sample of the decimation group.
In averaging mode, the value represents the sum of all ADC codes in the decimation group.

### Acquisition trigger message

| Bits    | Field name    | Description |
|---------|---------------|-------------|
| 63 : 56 | msg_type      | Fixed value 0x11 |
| 47 : 0  | timestamp     | Timestamp in units of 8 ns |

A trigger message is emitted when a triggered acquisition starts.
The trigger message is emitted before the first sample of the acquisition.

The trigger timestamp represents the moment when integration of the first sample starts.

### Acquisition overflow message

| Bits    | Field name    | Description |
|---------|---------------|-------------|
| 63 : 56 | msg_type      | Fixed value 0x40 |

An overflow message is emitted when the internal data buffer overflows.
It marks the point in the message sequence where an unknown number of messages have been discarded.


# Timetagger

The timetagger has 4 digital input signals.
If a rising or falling edge occurs on one of these signals, a timestamp is assigned to that event and a message is emitted and transferred via DMA.

A 4-cycle glitch filter is applied to the digital input signals.
This filter rejects digital pulses shorter than 4 clock cycles (32 ns).

Messages are only emitted for _enabled_ event types.
Rising edges and falling edges can be separately enabled or disabled for each digital input channel.
This is done via register `TIMETAGGER_EN`.

The messages are queued in a FIFO buffer inside the FPGA before being transferred via DMA.
This FIFO is necessary to hold messages for a short time while the DMA engine sets up a DMA transfer.
It has room for 4096 messages.

## Output data format

The output from the timetagger is a sequence of 64-bit messages.
Three types of messages are used:

* Event messages, emitted when a digital input event occurs that it enabled for time tagging.
* Marker messages, emitted on explicit request from software.
* Overflow messages, emitted when the internal data buffer overflows.

### Timetagger event message

| Bits    | Field name    | Description |
|---------|---------------|-------------|
| 63 : 60 | msg_type      | Fixed value 0x2 |
| 59 : 57 | channel       | Index of digital input channel where the event occurred. |
| 56      | falling_edge  | '1' if a falling edge occurred, '0' if a rising edge occurred. |
| 51 : 48 | input_state   | New state of digital input signals after the event. |
| 47 : 0  | timestamp     | Timestamp in units of 8 ns |

An event message is emitted when a digital input event occurs that is enabled for time tagging.

Note that multiple events can occur simultaneously.
If this happens, separate messages are emitted for all events that occurred.
These messages all have the same timestamp, and the same `input_state` field.

### Timetagger marker messages

| Bits    | Field name    | Description |
|---------|---------------|-------------|
| 63 : 56 | msg_type      | Fixed value 0x30 |
| 51 : 48 | input_state   | State of digital input signals. |
| 47 : 0  | timestamp     | Timestamp in units of 8 ns |

A marker message is emitted when software writes value 1 to register `TIMETAGGER_MARK`.
This may be used to establish synchronization between digital input events and some other type of events that are triggered or detected by software.

### Timetagger overflow message

| Bits    | Field name    | Description |
|---------|---------------|-------------|
| 63 : 56 | msg_type      | Fixed value 0x40 |

An overflow message is emitted when the internal data buffer overflows.
It marks the point in the message sequence where an unknown number of messages have been discarded.


# Registers

The behaviour of the FPGA is controlled via a set of registers.
These registers are accessible from the Zynq PS via memory mapped I/O.

The registers are located in a 2 MB region with address 0x43000000 to 0x431fffff in the PS address map.

Register access is implemented via an internal APB bus inside the FPGA.
This APB bus is connected via an intermediate AXI bus to port `M_AXI_GP0` of the Zynq PS-to-PL interface.

Within the register address space, the first 1 MB area is intended for various types of control and status registers that are accessed by user space software.
The second 1 MB area is intended for privileged registers that are only accessed by system software such as the Linux kernel driver.
The difference is that writing incorrect values to privileged registers can corrupt or crash the PS system, while writing to user registers can not.

The complete set of registers is listed below.
Access to undefined addresses within the register area must be avoided.
If such access occurs, the FPGA may either ignore it or map it to an existing register in a way that may be unexpected.

## Register list

This list specifies register addresses as offsets within the register address space.
The base address 0x43000000 must be added to obtain the absolute address on the PS memory bus.

**User registers**

| Address  | Name              | Description |
|----------|-------------------|-------------|
| 0x000000 | INFO              | Firmware version information |
| 0x000010 | IRQ_ENABLE        | Enable PL-to-PS interrupts |
| 0x000014 | IRQ_PENDING       | Mask of pending interrupt conditions |
| 0x000100 | DMA_EN            | Enable DMA transfers |
| 0x000104 | DMA_STATUS        | Status of DMA engine |
| 0x000108 | DMA_CLEAR         | Clear DMA error flags |
| 0x000180 | TIMESTAMP_LO      | Low 32 bits of global timestamp |
| 0x000184 | TIMESTAMP_HI      | High 16 bits of global timestamp |
| 0x000188 | TIMESTAMP_CLEAR   | Reset timestamp counter to zero |
| 0x000200 | ACQ_ADDR_START    | Start address of DMA buffer for analog acquisition |
| 0x000204 | ACQ_ADDR_END      | End address of DMA buffer for analog acquisition |
| 0x000208 | ACQ_ADDR_LIMIT    | Pause DMA when write pointer reaches this point |
| 0x00020C | ACQ_ADDR_INTR     | Raise interrupt when write pointer reaches thi point |
| 0x000210 | ACQ_ADDR_PTR      | Current DMA write pointer |
| 0x000214 | ACQ_DMA_CTRL      | Control DMA write stream for analog acquisition |
| 0x000218 | ACQ_INTR_CTRL     | Control DMA interrupts for analog acquisition |
| 0x00021C | ACQ_DMA_STATUS    | Status of DMA write stream for analog acquisition |
| 0x000220 | ACQUISITION_EN    | Enable analog data acquisition |
| 0x000224 | RECORD_LENGTH     | Number of samples per trigger minus 1. |
| 0x000228 | DECIMATION_FACTOR | Sample decimation factor minus 1. |
| 0x00022C | SHIFT_STEPS       | Nr of bit steps to shift accumulated value right |
| 0x000230 | AVERAGING_EN      | Select averaging or decimation |
| 0x000234 | CH4_MODE          | Enable 4-channel sampling (4-channel device only) |
| 0x000238 | SIMULATE_ADC      | Simulate ADC samples |
| 0x000240 | TRIGGER_MODE      | Trigger configuration |
| 0x000244 | TRIGGER_DELAY     | Trigger delay |
| 0x000248 | TRIGGER_STATUS    | Trigger status |
| 0x000280 | ADC_SAMPLE        | Most recent ADC samples from IN1 and IN2 |
| 0x000284 | ADC23_SAMPLE      | Most recent ADC samples from IN3 and IN4 (4-channel device only) |
| 0x00028C | ADC_RANGE_CLEAR   | Clear min/max ADC sample values |
| 0x000290 | ADC0_MINMAX       | Min/max ADC sample values from IN1 |
| 0x000294 | ADC1_MINMAX       | Min/max ADC sample values from IN2 |
| 0x000298 | ADC2_MINMAX       | Min/max ADC sample values from IN3 (4-channel device only) |
| 0x00029C | ADC3_MINMAX       | Min/max ADC sample values from IN4 (4-channel device only) |
| 0x000300 | TT_ADDR_START     | Start address of DMA buffer for timetagger |
| 0x000304 | TT_ADDR_END       | End address of DMA buffer for timetagger |
| 0x000308 | TT_ADDR_LIMIT     | Pause DMA when write pointer reaches this point |
| 0x00030C | TT_ADDR_INTR      | Raise interrupt when write pointer reaches thi point |
| 0x000310 | TT_ADDR_PTR       | Current DMA write pointer |
| 0x000314 | TT_DMA_CTRL       | Control DMA write stream for timetagger |
| 0x000318 | TT_INTR_CTRL      | Control DMA interrupts for timetagger |
| 0x00031C | TT_DMA_STATUS     | Status of DMA write stream for timetagger |
| 0x000320 | TIMETAGGER_EN     | Mask of enabled timetagger channels |
| 0x000324 | TIMETAGGER_MARK   | Emit marker record in timetagger data |
| 0x000404 | LED_STATE         | Control LEDs |

**Privileged registers**

| Address  | Name              | Description |
|----------|-------------------|-------------|
| 0x100000 | DMA_BUF_ADDR      | Base address of DMA buffer in RAM |
| 0x100004 | DMA_BUF_SIZE      | Size of DMA buffer in RAM |

## Register descriptions

Registers are 32 bits wide, but many registers use only a subset of the bits.
A _field_ is a range of meaningful bits within a register.
Some registers contain multiple fields.

Fields are marked as _RW_ or _RO_ or _WC_.

* RW fields can be modified by writing to the register.
  Reading from the register returns the most recent value written to the field.
* RO fields are read-only. Writes to these fields are ignored.
  The value of these fields typically changes spontaneously to reflect the status of the FPGA.
* WC fields are write-only and auto-clearing.
  Writing a 1 bit to the field triggers an action in the FPGA.
  Writing a 0 bit has no effect.
  WC fields are returned as 0 when reading from the register.

Bits that are not part of a field are _reserved_.
Reserved bits must be ignored when reading.
Reserved bits should be written as zero or left unmodified when writing.

### 0x000000: INFO

Firmware version information.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 7 : 0   | version_minor | RO     | Firmware minor version |
| 15 : 8  | version_major | RO     | Firmware major version |
| 23 : 16 | api_version   | RO     | Firmware API version, currently 0x01 |
| 31 : 24 |               | RO     | Fixed value 0x4a |

### 0x000010: IRQ_ENABLE

Enable PL-to-PS interrupts.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | irq_enable    | RW     | '1' to enable interrupts, '0' to disable |

This register acts as a master switch for interrupts from the FPGA.
Specific interrupt conditions can be separately enabled or disabled via other registers.

### 0x000014: IRQ_PENDING

Mask of pending interrupt conditions.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | acq_intr      | RO     | '1' if an interrupt is pending for analog acquisition DMA |
| 1       | tt_intr       | RO     | '1' if an interrupt is pending for timetagger DMA |

### 0x000100: DMA_EN

Enable DMA transfers.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | dma_en        | RW     | '1' to enable DMA transfers, '0' to disable |

This register acts as a master switch for the DMA engine in the FPGA.

Disabling DMA will prevent new bursts from starting, but allows ongoing bursts to complete.
Under normal circumstances, the completion of ongoing bursts can take at most a few microseconds.

### 0x000104: DMA_STATUS

Status of DMA engine.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | dma_busy      | RO     | '1' if a DMA burst is in progres |
| 1       | err_read      | RO     | '1' if a DMA read access failed |
| 2       | err_write     | RO     | '1' if a DMA write access failed |
| 3       | err_address   | RO     | '1' if access outside the DMA buffer was attempted |
| 4       | err_any       | RO     | '1' if any DMA error occurred |

If a DMA error occurs, no further DMA bursts will be started until the error is explicitly cleared.
The failed DMA burst will not be acknowledged to the subsystem that initiated it (analog or timetagger).
To recover from this situation, it will typically be necessary to stop and re-initialize all ongoing DMA data streams, clear the error, then restart DMA data streams.

An address error typically occurs when the DMA address range for the analog subsystem or the timetagger exceeds the size of the DMA buffer specified via `DMA_BUF_SIZE`.

### 0x000108: DMA_CLEAR

Clear DMA error flags.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | dma_clear     | WC     | Write '1' to clear pending DMA errors |

### 0x000180: TIMESTAMP_LO

Low 32 bits of global timestamp.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 0  |               | RO     | Least significant 32 bits of the global 8 ns timestamp counter. |

### 0x000184: TIMESTAMP_HI

High 16 bits of global timestamp.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 15 : 0  |               | RO     | Most significant 16 bits of the global 8 ns timestamp counter. |

When reading both `TIMESTAMP_LO` and `TIMESTAMP_HI`, note that the timestamp will change between the two reads, possibly resulting in an invalid combination of low and high part.
To avoid this, read `TIMESTAMP_HI`, then `TIMESTAMP_LO`, then `TIMESTAMP_HI` again.
Then check that the value of `TIMESTAMP_HI` has not changed during the sequence, otherwise retry.

### 0x000188: TIMESTAMP_CLEAR

Reset timestamp counter to zero.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | timestamp_clear | WC   | Write '1' to reset the timestamp counter. |

### 0x000200: ACQ_ADDR_START

Start address of DMA buffer for analog acquisition.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 7  |               | RW     | Start address as multiple of 128 bytes. |

The address must be aligned to a 128-byte boundary.
The address is interpreted as an offset relative to `DMA_BUF_ADDR`.

### 0x000204: ACQ_ADDR_END

End address of DMA buffer for analog acquisition.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 7  |               | RW     | End address as multiple of 128 bytes. |

The end address points past the end of the DMA buffer.
When the DMA write pointer reaches `ACQ_ADDR_END`, it immediately wraps back to `ACQ_ADDR_START`.

The address must be aligned to a 128-byte boundary.
The address is interpreted as an offset relative to `DMA_BUF_ADDR`.

### 0x000208: ACQ_ADDR_LIMIT

Pause DMA when write pointer reaches this point.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 7  |               | RW     | Address as multiple of 128 bytes. |

When the address pointer reaches the limit address, the DMA engine pauses further transfers until software updates the limit.

The address must be aligned to a 128-byte boundary.
The address is interpreted as an offset relative to `DMA_BUF_ADDR`.

### 0x00020C: ACQ_ADDR_INTR

Raise interrupt when write pointer reaches this point.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 3  |               | RW     | Address as multiple of 8 bytes. |

When the address pointer reaches this address, an interrupt occurs if analog acquisition interrupts are enabled.

The address must be aligned to an 8-byte boundary.
The address is interpreted as an offset relative to `DMA_BUF_ADDR`.

### 0x000210: ACQ_ADDR_PTR

Current DMA write pointer.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 3  |               | RO     | Address as multiple of 8 bytes. |

This register indicates the address where the next write transfer will start.
Write transfers have completed for locations up to (but excluding) this address.
This indicates that software can read valid data up to (but excluding) the pointer address.

The address will be aligned to an 8-byte boundary.
The address is interpreted as an offset relative to `DMA_BUF_ADDR`.

### 0x000214: ACQ_DMA_CTRL

Control DMA write stream for analog acquisition.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | acq_dma_en    | RW     | '1' to enable DMA transfers for analog acquisition. |
| 1       | acq_dma_init  | WC     | Write '1' to initialize the analog acquisition DMA stream |

Initializing the DMA stream resets the write pointer to `ACQ_ADDR_START` and discards pending data from the internal buffer.
Initializing is necessary after setting up the buffer address, and to recover from DMA errors.

### 0x000218: ACQ_INTR_CTRL

Control DMA interrupts for analog acquisition.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | acq_intr_en   | RW     | '1' to enable interrupt for DMA write pointer. |
| 1       | acq_intr_clear | WC    | Write '1' to clear a pending interrupt condition. |

### 0x00021C: ACQ_DMA_STATUS

Status of DMA write stream for analog acquisition.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | acq_dma_busy  | RO     | '1' if an uncompleted DMA burst is in progress. |

### 0x000220: ACQUISITION_EN

Enable analog data acquisition.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | acquisition_en | RW    | '1' to enable analog data acquisition. |

When analog acquisition is enabled, triggers are processed according to the configured trigger mode and samples are collected accordingly.
If analog acquisition is disabled, any ongoing sample collection stops immediately and further triggers are ignored.

### 0x000224: RECORD_LENGTH

Number of samples per trigger minus 1.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 15 : 0  | record_length | RW     | Number of samples per trigger minus 1. |

When a trigger condition occurs, the acquisition chain collects (`record_length` + 1) decimated samples.

### 0x000228: DECIMATION_FACTOR

Sample decimation factor minus 1.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 17 : 0  | decimation_factor | RW | Decimation factor minus 1. |

The ADCs sample at a fixed sample rate of 125 MSa/s.
The resulting sample stream is decimated by a factor (`decimation_factor` + 1).

If averaging is enabled, the sum of each group of (`decimation_factor` + 1) ADC samples is collected as a decimated sample.
If averaging is disabled, only the first sample of each group of (`decimation_factor` + 1) ADC samples is collected.

### 0x00022C: SHIFT_STEPS

Number of bit positions to shift accumulated value right.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 3 : 0   | shift_steps   | RW     | Number of bit positions. |

The accumulated value of each decimated sample is right-shifted by the configured number of bit positions before the value is transferred to DMA.
This is necessary to prevent overflow when a high decimation factor (above 1024) is used in averaging mode.

### 0x000230: AVERAGING_EN

Select averaging or decimation.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | averaging_en  | RW     | '1' to average samples, '0' to decimate samples. |

### 0x000234: CH4_MODE

Enable 4-channel sampling (only supported on 4-channel devices).

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | ch4_mode      | RW     | '1' to enable 4-channel sampling. |

When 4-channel sampling is disabled, only channels IN1 and IN2 are active.
One 64-bit word is emitted for each decimated sample, containing the data of these two channels.

When 4-channel sampling is enabled, channels IN3 and IN4 are also active.
Two 64-bit words are emitted for each decimated sample.
The first word contains the data for channels IN1 and IN2.
The second word contains the data for channels IN3 and IN4.

When 4-channel sampling is enabled, the decimation factor **must** be at least 2 (`DECIMATION_FACTOR` at least 1).

### 0x000238: SIMULATE_ADC

Simulate ADC samples.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | simulate_adc  | RW     | '1' to use simulated ADC samples, '0' to use real ADC samples. |

### 0x000240: TRIGGER_MODE

Trigger configuration.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | trig_auto_en  | RW     | '1' to enable auto-trigger mode. |
| 1       | trig_ext_en   | RW     | '1' to enable external triggering. |
| 5 : 4   | trig_ext_select | RW   | Select digital input channel to use as external trigger. |
| 7       | trig_ext_falling | RW  | '1' to trigger on falling edge, '0' to trigger on rising edge. |
| 8       | trig_force    | WC     | Write '1' to force a single trigger event. |

Auto-trigger mode takes precedence over external triggering.
If auto-triggering and external triggering are both disabled, no data acquisition takes place unless a trigger is forced by writing to `trig_force`.

### 0x000244: TRIGGER_DELAY

Trigger delay.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 15 : 0  | trigger_delay | RW     | Trigger delay in samples (8 ns). |

### 0x000248: TRIGGER_STATUS

Trigger delay.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | trig_waiting  | RO     | '1' if the acquisition chain is waiting for a trigger. |

### 0x000280: ADC_SAMPLE

Most recent ADC samples from IN1 and IN2.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 13 : 0  | adc0_sample   | RO     | ADC sample for channel IN1. |
| 29 : 16 | adc1_sample   | RO     | ADC sample for channel IN2. |

### 0x000284: ADC_SAMPLE

Most recent ADC samples from IN3 and IN4 (only supported on 4-channel devices).

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 13 : 0  | adc2_sample   | RO     | ADC sample for channel IN3. |
| 29 : 16 | adc3_sample   | RO     | ADC sample for channel IN4. |

### 0x00028C: ADC_RANGE_CLEAR

Most recent ADC samples from IN3 and IN4 (only supported on 4-channel devices).

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | adc_range_clear | WC   | Write '1' to clear min/max ADC sample values. |

### 0x000290: ADC0_MINMAX

Minimum and maximum ADC sample from IN1.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 13 : 0  | adc0_min      | RO     | Minimum ADC sample from IN1 since range cleared. |
| 29 : 16 | adc0_max      | RO     | Maximum ADC sample from IN1 since range cleared. |

### 0x000294: ADC1_MINMAX

Minimum and maximum ADC sample from IN2.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 13 : 0  | adc1_min      | RO     | Minimum ADC sample from IN2 since range cleared. |
| 29 : 16 | adc1_max      | RO     | Maximum ADC sample from IN2 since range cleared. |

### 0x000298: ADC2_MINMAX

Minimum and maximum ADC sample from IN3 (only supported on 4-channel devices).

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 13 : 0  | adc2_min      | RO     | Minimum ADC sample from IN3 since range cleared. |
| 29 : 16 | adc2_max      | RO     | Maximum ADC sample from IN3 since range cleared. |

### 0x00029C: ADC3_MINMAX

Minimum and maximum ADC sample from IN4 (only supported on 4-channel devices).

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 13 : 0  | adc3_min      | RO     | Minimum ADC sample from IN4 since range cleared. |
| 29 : 16 | adc3_max      | RO     | Maximum ADC sample from IN4 since range cleared. |

### 0x000300: TT_ADDR_START

Start address of DMA buffer for timetagger.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 7  |               | RW     | Start address as multiple of 128 bytes. |

The address must be aligned to a 128-byte boundary.
The address is interpreted as an offset relative to `DMA_BUF_ADDR`.

### 0x000304: TT_ADDR_END

End address of DMA buffer for timetagger.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 7  |               | RW     | End address as multiple of 128 bytes. |

The end address points past the end of the DMA buffer.
When the DMA write pointer reaches `TT_ADDR_END`, it immediately wraps back to `TT_ADDR_START`.

The address must be aligned to a 128-byte boundary.
The address is interpreted as an offset relative to `DMA_BUF_ADDR`.

### 0x000308: TT_ADDR_LIMIT

Pause DMA when write pointer reaches this point.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 7  |               | RW     | Address as multiple of 128 bytes. |

When the address pointer reaches the limit address, the DMA engine pauses further transfers until software updates the limit.

The address must be aligned to a 128-byte boundary.
The address is interpreted as an offset relative to `DMA_BUF_ADDR`.

### 0x00030C: TT_ADDR_INTR

Raise interrupt when write pointer reaches this point.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 3  |               | RW     | Address as multiple of 8 bytes. |

When the address pointer reaches this address, an interrupt occurs if analog acquisition interrupts are enabled.

The address must be aligned to an 8-byte boundary.
The address is interpreted as an offset relative to `DMA_BUF_ADDR`.

### 0x000310: TT_ADDR_PTR

Current DMA write pointer.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 3  |               | RO     | Address as multiple of 8 bytes. |

This register indicates the address where the next write transfer will start.
Write transfers have completed for locations up to (but excluding) this address.
This indicates that software can read valid data up to (but excluding) the pointer address.

The address will be aligned to an 8-byte boundary.
The address is interpreted as an offset relative to `DMA_BUF_ADDR`.

### 0x000314: TT_DMA_CTRL

Control DMA write stream for timetagger.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | tt_dma_en     | RW     | '1' to enable DMA transfers timetagger. |
| 1       | tt_dma_init   | WC     | Write '1' to initialize the timetagger DMA stream |

Initializing the DMA stream resets the write pointer to `TT_ADDR_START` and discards pending data from the internal buffer.
Initializing is necessary after setting up the buffer address, and to recover from DMA errors.

### 0x000318: TT_INTR_CTRL

Control DMA interrupts for timetagger.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | tt_intr_en    | RW     | '1' to enable interrupt for DMA write pointer. |
| 1       | tt_intr_clear | WC     | Write '1' to clear a pending interrupt condition. |

### 0x00031C: TT_DMA_STATUS

Status of DMA write stream for timetagger.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       | tt_dma_busy   | RO     | '1' if an uncompleted DMA burst is in progress. |

### 0x000320: TIMETAGGER_EN

Mask of enabled timetagger channels.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       |               | RW     | '1' to enable rising edge events on digital input 0. |
| 1       |               | RW     | '1' to enable falling edge events on digital input 0. |
| 2       |               | RW     | '1' to enable rising edge events on digital input 1. |
| 3       |               | RW     | '1' to enable falling edge events on digital input 1. |
| ...     |               | RW     | ... |
| 7       |               | RW     | '1' to enable falling edge events on digital input 3. |

### 0x000324: TIMETAGGER_MARK

Emit marker record in timetagger data.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 0       |               | WC     | Write '1' to emit a marker record in the timetagger data stream. |

### 0x000404: LED_STATE

Control LEDs.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 7 : 4   |               | RW     | Mask controlling 4 yellow LEDs on the side of the Red Pitaya. |

### 0x100000: DMA_BUF_ADDR

Base address of DMA buffer in RAM.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 12 |               | RW     | Base address of the DMA buffer in physical RAM. |

This register controls which region of PS RAM may be accessed via DMA.
Typically, the Linux kernel driver must allocate a suitable range of physical memory and initialize these registers accordingly.

The base address must be aligned to a 4 kB boundary.

### 0x100004: DMA_BUF_SIZE

Size of DMA buffer in RAM.

| Bits    | Field name    | Access | Description |
|---------|---------------|--------|-------------|
| 31 : 12 |               | RW     | DMA buffer size. |

If this register is set to zero, no DMA access is allowed.


# DMA

The FPGA acts as an AXI bus master to transfer bulk data to system RAM.
This data transfer pattern is commonly referred to as DMA.
Note that the Zynq DMA controller (DMAC) is not involved in these data transfers in any way.

DMA access is implemented via an internal AXI bus master inside the FPGA, connected to port `S_AXI_HP0` of the Zynq PS-to-PL interface.

## Data transfer pattern

The AXI HP data bus is 64 bits wide.
All DMA transfers use the full width of the bus on every beat.
Therefore, the smallest unit of data transferred via DMA is a 64-bit word.
The size of every DMA transfer is a multiple of 8 bytes, and the address of every transfer is aligned to a multiple of 8 bytes.

(This is an implementation choice in the firmware.
It is possible in principle to transfer smaller amounts of data via the AXI bus, but the PuzzleFW firmware is designed to transfer 64-bit words in all cases.)

Data are temporarily queued in a FIFO RAM block inside the FPGA until the DMA engine is ready to start a transfer.
This is necessary because DMA operates in bursts, and it may take some time before the DMA engine can initiate a burst.
The two DMA data streams (analog acquisition, timetagger) each have a separate FIFO memory.

If sufficient data are queued up inside the FPGA, the DMA engine transfers 16-beat bursts (blocks of 128 bytes).
This is the most efficient way to use the AXI bus.
If the amount of available data is insufficient for a full burst, the DMA engine waits briefly (2 microseconds) for more data to arrive.
After waiting, if there are still insufficient data for a full burst, the available data are transferred via single-beat bursts (blocks of 8 bytes).

The highest sustainable data transfer rate is approximately 105,000,000 64-bit words per second, or 800 MiB per second.

## Global DMA buffer

A global address window determines the range of allowed target addresses for DMA transfers.
This window is configured through the pair of registers `DMA_BUF_ADDR` and `DMA_BUF_SIZE`.

Whenever the FPGA initiates a DMA transfer, it first checks that the target address is within the configured buffer.
If the transfer falls outside the buffer, it fails and triggers an _address error_.

The start and size of the DMA buffer must be aligned to 4 kB page boundaries.

## Per-stream buffer segments

For each of the two DMA data streams (analog acquisition, timetagger) a dedicated segment may be allocated within the global DMA buffer.
These segments are used as circular buffers.

All addresses that specify the start or end of a buffer segment, or any location within a buffer, are specified as offsets relative to the start of the global DMA buffer `DMA_BUF_ADDR`.

For the analog aqcuisition data stream, the start and end of the buffer segment are configured through registers `ACQ_ADDR_START` and `ACQ_ADDR_END`, with `ACQ_ADDR_END` pointing to the first byte past the end of the segment.
The start and end addresses must be aligned to 128-byte boundaries.

The _write pointer_ of the DMA stream is stored in register `ACQ_ADDR_PTR`.
It indicates the destination address of the next DMA transfer.
It also signals to software that write transfers have completed for the region up to (but excluding) the write pointer.
The write pointer is read-only for software.
The FPGA automatically updates the pointer after each data transfer.
If the write pointer reaches the end of the segment, it immediately wraps back to the start.
The write pointer is always aligned to an 8-byte boundary.

If the write pointer reaches the address configured in `ACQ_ADDR_LIMIT`, no further DMA transfers will occur until the limit is updated.
Software can use this limit to prevent the DMA engine from overwriting parts of the buffer that have not yet been procecessed.
The limit address must be aligned to a 128-byte boundary.

If the write pointer reaches the address configured in `ACQ_ADDR_INTR`, an interrupt is triggered.
This happens only if the corresponding interrupt is enabled in registers `IRQ_ENABLE` and `ACQ_INTR_CTRL`.
The interrupt address must be aligned to an 8-byte boundary.

The timetagger data stream works in the same way as the analog acquisition data stream, but uses a separate set of registers.

## Cache coherence

Keep in mind that the Zynq architecture does not provide transparent cache coherence.
When the FPGA writes data to system RAM via DMA, the cache of the ARM processor may still contain obsolete data for the same address range.

Software running on the Zynq PS that wants to read from the DMA buffer, must either take explicit steps to invalidate the cache, or it must configure the DMA buffer as non-cacheable memory.


# Interrupts

The FPGA can send interrupts to the Zynq PS to notify the software of specific conditions.
Interrupts are routed via `IRQ_F2P[0]`, which corresponds to interrupt 61 in the Zynq Generic Interrupt Controller (GIC).

Two types of conditions can trigger an interrupt:

* DMA transfers for analog acquisition have progressed to a configurable point.
* DMA transfers for the timetagger have progressed to a configurable point.

Each type of condition has a separate interrupt enable switch, configured via registers `ACQ_INTR_CTRL` and `TT_INTR_CTRL`.
Furthermore, there is a master interrupt enable switch, configured via register `IRQ_ENABLE`.

For example, interrupts may be used to wait for a certain amount of data from the analog acquisition system as follows:

* Write register `ACQ_ADDR_INTR` to specify the amount of progress after which it wants to be notified.
* Write register `ACQ_INTR_CTRL` to clear any pending interrupt and enable interrupts on DMA progress.
* Read register `ACQ_ADDR_PTR` to check whether DMA progress has already passed the configured threshold.
  This is necessary to avoid a race condition where software keeps waiting for a condition thas has already happened.
* Write register `IRQ_ENABLE` to enable interrupts.
* Wait for the interrupt or do other stuff in the mean time.
* When the interrupt occurs, the interrupt handler may write register `IRQ_ENABLE` to disable interrupts until the software has time to handle the cause of the interrupt.
* Optionally read register `IRQ_PENDING` to confirm that the interrupt was caused by analog data acquisition.
* Write register `ACQ_INTR_CTRL` to clear the interrupt and disable further interrupts of this type.
* Process the data that arrived in the DMA buffer.

Interrupts can be used to wait for timetagger data in a similar way.