Practical FPGA Development on the Alveo U250: Vector Addition and Falcon Graph Vector Search

I recently started exploring FPGA hardware for my research and ran reproduction experiments with Falcon graph vector search on an Alveo U250. This post collects the development workflow, notes on the source code, and issues encountered during those experiments as a starting point for trying my own designs.

Moving a CPU algorithm to an FPGA requires rethinking how data flows: how inputs reach the card, whether computation can accept data continuously, where random accesses cause stalls, and how much of a query’s time goes into transfers. A small operator provides a useful starting point for checking correctness and measuring execution before changing the compute and memory structure.

This article uses vector addition to connect the host, onboard DDR, and compute units on an Alveo U250. It then examines random access, candidate queues, and pipelines with feedback through Falcon graph vector search. The examples use Linux, Vitis 2023.2, and the corresponding XRT runtime, with command-line builds and execution.

FPGA Hardware and Programming

An FPGA, or Field-Programmable Gate Array, contains configurable logic cells, registers, and interconnects. Configuration data determines the logic these cells implement and how they connect. Once a design is complete, the tools generate a bitstream that configures the device to implement the circuit. In the U250 acceleration workflow, the runtime loads this configuration through an .xclbin file.

USTC Vlab’s FPGA Principles guide introduces lookup tables and programmable interconnects, starting from logic gates.¹ A lookup table, or LUT, stores the truth table of a Boolean function; the input signals select the corresponding output. A two-input AND gate, for example, needs four stored results for its four possible input combinations. Flip-flops hold state at clock edges, connecting combinational logic into a circuit that advances each cycle. The guide illustrates slices and CLBs using Spartan-6. The U250 uses a later UltraScale+ device, whose resource organization is described in its device documentation.

Practical designs also use dedicated hardware resources. DSP blocks support multiplication and multiply-add operations, BRAM and URAM provide on-chip storage, and LUTs and flip-flops implement combinational logic, control state, and some buffering. In the graph search example, distance computation, candidate queues, Bloom filters, and FIFOs all map to these resources. Larger queues consume more storage or logic, while the number and layout of memory ports affect which modules can operate concurrently. These relationships help explain changes in LUT, FF, DSP, BRAM, and URAM usage in synthesis reports.

Pipelining describes how computation proceeds through these resources. Consider a computation that reads inputs, multiplies, adds, and writes results. Hardware can organize these operations into stages, with different inputs occupying different stages at the same time. Latency is the number of cycles an input takes to pass through the pipeline. The initiation interval, or II, is the minimum interval between the starts of successive iterations. A pipeline with a latency of 20 cycles and an II of 1 can accept a new input every cycle when inputs are available and downstream stages are not blocked. Each input still takes 20 cycles to produce a result. II therefore directly affects the sustained processing rate of a loop.

The number of parallel lanes determines how much work can be processed at once. Replicating an addition datapath into 16 lanes allows 16 pairs of elements to be added concurrently, but the memory interface must supply all 16 pairs and accept their results. Pipelining overlaps iterations; loop unrolling allows operations from multiple iterations to be implemented in parallel. Both are constrained by data dependencies, compute resources, and memory access capacity.² The vector addition example combines PIPELINE, UNROLL, and 512-bit interfaces to show how they work together.

You can describe this datapath directly in a hardware description language such as Verilog or SystemVerilog, or use high-level synthesis (HLS) to generate a register-transfer-level (RTL) design from synthesizable C/C++. AMD’s Vitis HLS 2023.2 tutorial covers the path from C/C++ to RTL and exporting a Vitis kernel.³ HLS schedules operations, allocates resources, and generates control logic according to dependencies and constraints. Arrays map to storage, function arguments become hardware interfaces, and loop operations either share resources or execute in parallel according to the schedule.

HLS provides a starting point for experiments with existing C/C++ algorithms, but kernel code must meet synthesis constraints. Storage sizes usually need to be known at synthesis time, and software patterns such as dynamic allocation and arbitrary recursion cannot be carried over directly. The host program can still use ordinary C++, dynamic memory, filesystems, and runtime libraries. In this example, host.cpp prepares data and submits work, while vadd.cpp defines the computation on the FPGA.

The U250 connects to a server through PCIe. The CPU prepares the inputs, while XRT manages the device, buffers, and task submission. DMA transfers data between host memory and onboard DDR. The FPGA kernel reads its inputs from DDR and writes the results back. The platform supplies infrastructure such as PCIe and memory controllers through interfaces exposed to application kernels. AXI is the interface protocol used to carry data and control information between modules. In the code below, m_axi lets the kernel initiate memory reads and writes, while s_axilite carries addresses, scalar arguments, and control information such as the start signal.

Turning source code into loadable hardware also requires synthesis, placement, and routing. Synthesis maps logic to device resources, placement assigns their locations, routing connects them, and timing analysis checks whether signals arrive within the clock period. A full hardware build can take substantial time, so development usually starts with functional simulation, followed by HLS schedule and resource reports, implementation, and measurements on the card. Graph vector search adds random reads, repeated visits, and dependencies between candidate queues and compute modules. A simple operator helps establish the workflow before examining these issues in Falcon.

Setting Up the U250 Development Environment

The U250 uses onboard DDR4 memory. Programming it involves both PCIe transfers between the host and card and accesses between compute units and DDR banks. More parallel compute lanes increase demand for DDR bandwidth; the reduction in total request time also depends on how much time PCIe transfers consume.

Before compiling, check that Vitis, XRT, and the development platform are compatible with the card’s shell. An .xpfm file describes the build platform, while an .xclbin contains the application kernels and their connectivity. Consult the platform release notes for shell compatibility.⁴ AMD’s Vitis 2023.2 hello_world example lists xilinx_u250_gen3x16_xdma_4_1_202210_1 as its default U250 platform.⁵ Use that identifier to locate the platform package and check it against the shell installed on your server.

The commands below use the following tools and platform:

• Development tools: Vitis/Vivado 2023.2.
• Runtime: XRT 2.16.204, from the 2023.2 branch.
• Development platform: xilinx_u250_gen3x16_xdma_4_1_202210_1.

Load the tool environment and inspect the device, adjusting paths and the PCIe bus/device/function (BDF) address for your server. xbutil validate runs tests on the card, so run it when the device is idle.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

source /path/to/Xilinx/Vitis/2023.2/settings64.sh
source /opt/xilinx/xrt/setup.sh

v++ --version
xbutil examine
export DEVICE_BDF=0000:01:00.1  # Use the BDF reported by xbutil examine
xbutil examine --device "$DEVICE_BDF" --report platform
xbutil validate --device "$DEVICE_BDF"

export PLATFORM=/opt/xilinx/platforms/xilinx_u250_gen3x16_xdma_4_1_202210_1/xilinx_u250_gen3x16_xdma_4_1_202210_1.xpfm
platforminfo --platform "$PLATFORM"

Save the shell identifier, platform path, Vitis/XRT versions, driver status, and PCIe link status with the inspection results. The physical length of a PCIe slot does not establish the negotiated link speed and width. If DMA is much slower than expected, check for a reduced link speed or width. The example host program below selects device index 0. On a server with multiple cards, change this to the target device’s index and verify its correspondence with the BDF.

Vector Addition with HLS and XRT

Start with c[i] = a[i] + b[i] using 32-bit unsigned integers, with addition wrapping modulo $2^{32}$. Reading two arrays and writing one makes the result easy to check and provides a clear way to observe the effect of memory configurations.

Save the following code as vadd.cpp. Each ap_uint<512> holds 16 elements. The outer loop processes one wide word, while the inner loop is unrolled into 16 addition lanes. The three m_axi bundles create independent AXI master interfaces; the link configuration specifies their DDR connections.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

#include <ap_int.h>

extern "C" void vadd(const ap_uint<512>* a,
                     const ap_uint<512>* b,
                     ap_uint<512>* c,
                     unsigned int nwords) {
#pragma HLS INTERFACE m_axi port=a offset=slave bundle=gmem0
#pragma HLS INTERFACE m_axi port=b offset=slave bundle=gmem1
#pragma HLS INTERFACE m_axi port=c offset=slave bundle=gmem2
#pragma HLS INTERFACE s_axilite port=a bundle=control
#pragma HLS INTERFACE s_axilite port=b bundle=control
#pragma HLS INTERFACE s_axilite port=c bundle=control
#pragma HLS INTERFACE s_axilite port=nwords bundle=control
#pragma HLS INTERFACE s_axilite port=return bundle=control

    for (unsigned int w = 0; w < nwords; ++w) {
#pragma HLS PIPELINE II=1
        const ap_uint<512> aw = a[w];
        const ap_uint<512> bw = b[w];
        ap_uint<512> cw = 0;
        for (int lane = 0; lane < 16; ++lane) {
#pragma HLS UNROLL
            const ap_uint<32> av = aw.range(32 * lane + 31, 32 * lane);
            const ap_uint<32> bv = bw.range(32 * lane + 31, 32 * lane);
            const ap_uint<32> sum = av + bv;
            cw.range(32 * lane + 31, 32 * lane) = sum;
        }
        c[w] = cw;
    }
}

PIPELINE II=1 requests that a new outer-loop iteration start every cycle. The synthesis report shows the achieved initiation interval; external memory stalls also affect throughput on the card. This kernel runs at 300 MHz, which you can inspect with xclbinutil --info --input build/hw/vadd.xclbin. At one 64-byte word per cycle, the theoretical bandwidth of each port is $64\times300\times10^6=19.2$ GB/s, equal to the theoretical bandwidth of one DDR4-2400 channel. The three ports together provide a theoretical 57.6 GB/s. Memory stalls reduce the achieved throughput.

Save the following configuration as connectivity.cfg. It assumes that the platform exposes DDR[0], DDR[1], and DDR[2]; check these names in the platform information.

1
2
3
4
5

[connectivity]
nk=vadd:1:vadd_1
sp=vadd_1.a:DDR[0]
sp=vadd_1.b:DDR[1]
sp=vadd_1.c:DDR[2]

Separate bundles give the three accesses their own AXI interfaces, and mapping them to different banks distributes the load across memory controllers. Later, change only this configuration to place all three arguments in DDR[0] and compare execution times with the same compute logic. This experiment shows whether bank contention limits the application.

The host pads a partial group of 16 elements with zeros and allocates space for complete wide words. The kernel’s nwords argument counts wide words, while the input length counts elements. Confusing these units can cause out-of-bounds accesses or leave most of the input unprocessed.

Save the following code as host.cpp. The program accepts an .xclbin path, an element count, and a repetition count. It checks every valid output element after each execution and prints the timing for each measured iteration. Large unsigned input values also exercise addition wraparound.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80

#include <xrt/xrt_bo.h>
#include <xrt/xrt_device.h>
#include <xrt/xrt_kernel.h>

#include <chrono>
#include <cstddef>
#include <cstdint>
#include <exception>
#include <iostream>
#include <stdexcept>
#include <string>

std::size_t positive_count(const char* arg, std::size_t limit) {
    const std::string text(arg);
    if (text.empty() || text.find_first_not_of("0123456789") != std::string::npos)
        throw std::invalid_argument("expected a positive integer");
    const auto value = std::stoull(text);
    if (value == 0 || value > limit)
        throw std::out_of_range("argument outside supported range");
    return static_cast<std::size_t>(value);
}

int main(int argc, char** argv) {
    try {
        if (argc != 4) {
            std::cerr << "usage: ./host kernel.xclbin elements repetitions\n";
            return 1;
        }
        const auto n = positive_count(argv[2], std::size_t{1} << 24);
        const auto repetitions = positive_count(argv[3], 10000);
        const auto nwords = static_cast<unsigned int>((n + 15) / 16);
        const auto padded = static_cast<std::size_t>(nwords) * 16;
        const auto bytes = padded * sizeof(std::uint32_t);

        auto device = xrt::device(0);
        const auto uuid = device.load_xclbin(argv[1]);
        auto kernel = xrt::kernel(device, uuid, "vadd");
        auto a = xrt::bo(device, bytes, kernel.group_id(0));
        auto b = xrt::bo(device, bytes, kernel.group_id(1));
        auto c = xrt::bo(device, bytes, kernel.group_id(2));
        auto* pa = a.map<std::uint32_t*>();
        auto* pb = b.map<std::uint32_t*>();
        auto* pc = c.map<std::uint32_t*>();
        for (std::size_t i = 0; i < padded; ++i) {
            pa[i] = i < n ? std::uint32_t{0xfffffff0u} +
                               static_cast<std::uint32_t>(i) : 0;
            pb[i] = i < n ? static_cast<std::uint32_t>(3 * i + 7) : 0;
        }

        using Clock = std::chrono::steady_clock;
        const auto ms = [](auto begin, auto end) {
            return std::chrono::duration<double, std::milli>(end - begin).count();
        };
        std::cout << "iteration,h2d_ms,submit_wait_ms,d2h_ms,request_ms\n";
        // Iteration zero warms up the execution path and is excluded from output.
        for (std::size_t rep = 0; rep <= repetitions; ++rep) {
            const auto t0 = Clock::now();
            a.sync(XCL_BO_SYNC_BO_TO_DEVICE);
            b.sync(XCL_BO_SYNC_BO_TO_DEVICE);
            const auto t1 = Clock::now();
            auto run = kernel(a, b, c, nwords);
            run.wait();
            const auto t2 = Clock::now();
            c.sync(XCL_BO_SYNC_BO_FROM_DEVICE);
            const auto t3 = Clock::now();
            for (std::size_t i = 0; i < n; ++i) {
                const auto expected = static_cast<std::uint32_t>(pa[i] + pb[i]);
                if (pc[i] != expected)
                    throw std::runtime_error("mismatch at element " + std::to_string(i));
            }
            if (rep != 0)
                std::cout << rep << ',' << ms(t0, t1) << ',' << ms(t1, t2)
                          << ',' << ms(t2, t3) << ',' << ms(t0, t3) << '\n';
        }
        return 0;
    } catch (const std::exception& e) {
        std::cerr << e.what() << '\n';
        return 1;
    }
}

group_id() selects a buffer’s memory group from the kernel argument connectivity, keeping host allocation consistent with the linked design. map() returns a host-side mapped address. After writing the mapped region, call sync(TO_DEVICE) to make the updated input available to the FPGA. Once the kernel finishes, call sync(FROM_DEVICE) before reading the output on the host. XRT object lifetimes manage device and buffer resources.⁶

The host limits inputs to $2^{24}$ elements to bound memory use. Before increasing this limit for a real dataset, check the range of the element-count type, the combined size of the three buffers, and the available capacity of each bank.

Building, Emulating, and Running on the Card

Compile the host program in the project directory, then build the kernel for software emulation. v++ -c produces an .xo file, and v++ -l produces an .xclbin using the platform and connectivity configuration. Separate directories for each target help avoid accidentally reusing old build artifacts.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

g++ -std=c++17 -O2 -Wall -Wextra host.cpp \
  -I"$XILINX_XRT/include" -L"$XILINX_XRT/lib" \
  -Wl,-rpath,"$XILINX_XRT/lib" -lxrt_coreutil -pthread -o host

export TARGET=sw_emu
mkdir -p "build/$TARGET"
v++ -c -t "$TARGET" --platform "$PLATFORM" -k vadd \
  --temp_dir "build/$TARGET/compile" \
  -o "build/$TARGET/vadd.xo" vadd.cpp
v++ -l -t "$TARGET" --platform "$PLATFORM" \
  --config connectivity.cfg --temp_dir "build/$TARGET/link" \
  -o "build/$TARGET/vadd.xclbin" "build/$TARGET/vadd.xo"
emconfigutil --platform "$PLATFORM" --od "build/$TARGET"

export EMCONFIG_PATH="$PWD/build/$TARGET"
XCL_EMULATION_MODE=sw_emu ./host "build/$TARGET/vadd.xclbin" 1003 2

An input length of 1003 exercises tail padding. Also test lengths of 1, 15, 16, 17, and 1024 to cover inputs shorter than one wide word, word boundaries, and sequential accesses. Software emulation checks host arguments, data encoding, and algorithm outputs. Runs on the card provide throughput and latency measurements.

Set TARGET to hw_emu, repeat the compilation, linking, and emconfigutil commands, and set XCL_EMULATION_MODE to hw_emu for execution. Hardware emulation includes RTL behavior and helps inspect interfaces and blocked dataflow. Begin with short inputs, since large datasets substantially increase emulation time. After adding FIFOs and feedback loops, examine channel occupancy and stalls for possible blocking.

For the hardware build, set TARGET to hw and repeat compilation and linking. This target does not need emconfigutil. Once the device is idle, run:

1
2

unset XCL_EMULATION_MODE
./host build/hw/vadd.xclbin 1048576 100 > timings.csv

Loading an .xclbin changes the application running on the device, so coordinate it with other jobs on the server. Builds can run on a separate development machine; loading and measurement require an available card.

After synthesis, inspect the achieved II, interface widths, and resource usage, followed by the timing report after placement and routing. HLS provides a clock estimate, while the implementation report reflects the routed design. If II=1 is not achieved, the report usually identifies a dependency, memory-port, or compute-resource constraint. If the target II is achieved but the implementation frequency drops, inspect critical paths and placement congestion.

Measuring Performance and Optimizing Memory Access

The host code divides a request into host-to-device (H2D) transfer, submission and waiting, and device-to-host (D2H) transfer. When interpreting the results, distinguish these timing scopes:

• submit_wait_ms measures the interval from kernel submission until the host finishes waiting for completion. It includes kernel execution time, XRT submission overhead, and host waiting overhead.
• Device-side kernel time comes from Vitis analysis with the corresponding monitoring configuration.
• request_ms spans input synchronization through output synchronization. It excludes device initialization, bitstream loading, allocation, input generation, and correctness checks.

For $N$ elements, the algorithm reads and writes $12N$ useful bytes; actual transfers also include padding in the final wide word. Dividing $12N$ by device-side time $T_k$ gives the kernel’s effective throughput. Dividing $N$ by request time $T_r$ gives the element processing rate for the complete request. GB/s here means $10^9$ bytes per second. Startup overhead can dominate small inputs; larger inputs make sustained memory throughput easier to observe.

The measurements used Vitis 2023.2 and XRT 2.16.204, with exclusive use of one U250 during execution. Both configurations, one bank per argument and all arguments in DDR[0], passed correctness checks for lengths 1003, 1, 15, 16, 17, and 1024, covering tail padding and wide-word boundaries.

The table reports host-side times in milliseconds. Values are means, with minima and maxima in parentheses. The two large inputs were repeated 100 and 20 times, respectively, and the single-element input was repeated twice. All statistics exclude the initial warmup run.

Dividing the useful traffic volume, $12N$, by the submission-and-wait time gives the following effective bandwidth:

The comparison shows three effects:

• Bank contention limits throughput. At roughly 16 million elements, the submission-and-wait time with everything in DDR[0] is about 2.7 times that of the three-bank configuration. The three-bank configuration achieves an aggregate 43.3 GB/s, or about 14.4 GB/s per port, reaching roughly 75% of the theoretical aggregate bandwidth.
• Transfers account for most of the complete request. At the same input size, the three-bank request takes 27.628 ms, of which H2D and D2H together take about 23 ms, or 83%. Using the byte counts of two input arrays and one output array gives approximately 8.4 GB/s for H2D and 9.6 GB/s for D2H.
• Fixed overhead affects small requests. Processing a single element still takes about 48 µs for submission and waiting and about 108 µs for the complete request. Larger inputs amortize this overhead across more elements.

After comparing bank mappings, you can test multiple compute units. When adding CUs, check each CU’s input and output mappings, whether the host submits work concurrently, and whether DDR is already saturated. Two CUs sharing a busy bank may deliver nearly the same aggregate throughput despite the additional resources. Bank traffic and CU activity measurements can help explain that result.

If an application repeatedly uses the same index or model, upload it once during initialization and send only queries and results in the query loop. Record initialization time and steady-state query time separately. For independent data blocks, multiple buffer sets can overlap the transfer of the next block with computation on the current one. The host must track transfer and execution completion for each buffer set, and the execution timeline should confirm that overlap occurs.

Splitting a kernel into read, compute, and write functions allows DATAFLOW to connect them with streams and overlap their execution. FIFO depth absorbs temporary differences in stage rates. If a consumer consistently falls behind its producer, the FIFO eventually fills, so the slower stage needs more throughput. A pipeline with feedback also needs a termination protocol: all modules must agree on the final request, final result, and end-of-query signal. Otherwise, some modules may exit while others continue waiting for data. Falcon’s graph search provides a concrete example of such a pipeline.

Falcon Query Processing and Source Code

Falcon is the FPGA graph search system presented in Fast Graph Vector Search via Hardware Acceleration and Delayed-Synchronization Traversal.⁷ The repository’s FPGA_single_DDR directory retains a development-stage implementation that is useful for understanding the modules. The multi-DDR inter-query v1.3 and intra-query v1.5 designs support local experiments. The reproduction workflow below uses the intra-query version.

Graph vector search represents vectors as nodes and connects nearby vectors with edges. A query starts at an entry node, selects candidates by distance, and expands their neighbors to find nodes close to the query. HNSW, or Hierarchical Navigable Small World, organizes this search through multiple graph layers. Recall@10 measures how many of the true ten nearest neighbors appear in the ten returned results: divide their overlap by ten for each query, then average over the query set.

The following tree lists the main files used for reading and running the implementation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

Falcon-accelerate-graph-vector-search/
├── FPGA_single_DDR/
│   └── FPGA_single_DDR_single_layer_v1.3_opt_queue/
│       └── src/
│           ├── host.cpp                          # Host data loading and task submission
│           ├── vadd.cpp                          # Kernel entry point and top-level dataflow
│           ├── per_channel_processing_wrapper.hpp # Connects the per-channel processing modules
│           ├── scheduler.hpp                     # Candidate scheduling and work in flight
│           ├── DRAM_utils.hpp                    # Adjacency-list and vector memory access
│           ├── bloom_fetch_compute.hpp           # Connects filtering, fetching, and distance computation
│           ├── priority_queue.hpp                # Priority queue implementation
│           └── constants.hpp                     # Dimensions, queue capacities, and synthesis parameters
├── FPGA_multi_DDR/
│   └── FPGA_intra_query_v1.5_support_batching_longer_FIFO/
│       ├── Makefile                              # Host and kernel builds
│       ├── connectivity.cfg                      # DDR bank mapping and clock configuration
│       └── src/
│           ├── host.cpp                          # Multi-channel data loading, execution, and validation
│           ├── vadd.cpp                          # Kernel entry point for intra-query parallelism
│           ├── DRAM_utils.hpp                    # Maps node IDs to channels and addresses
│           └── constants.hpp                     # Channel count, dimensions, and queue capacities
├── vector_search_baselines/
│   └── scripts_hnsw/
│       ├── README_hnsw.md                        # Index construction and conversion instructions
│       ├── construct_and_search_hnsw.py           # Builds and searches a CPU HNSW index
│       └── hnsw_to_FPGA.py                        # Exports the index in the FPGA format
└── perf_test_scripts/
    ├── README.md                                 # Experiment script instructions
    ├── perf_test_throughput.py                    # Throughput experiments
    ├── perf_test_latency.py                       # Latency experiments
    └── utils.py                                  # Report parsing and result processing

Open FPGA_single_DDR/FPGA_single_DDR_single_layer_v1.3_opt_queue/src/vadd.cpp. The function retains the name vadd, but its arguments now include queries, entry nodes, graph edges, database vectors, and search queue settings. At the top level, DATAFLOW connects read_queries, per_channel_processing_wrapper, and write_results. Query batches enter the compute module, which produces IDs, distances, and debug information. The writeback module uses s_finish_batch to tell the reader to continue.⁸

In per_channel_processing_wrapper.hpp, follow the candidate flow through the processing modules. scheduler.hpp manages candidates awaiting expansion and work in flight; DRAM_utils.hpp reads adjacency lists; bloom_fetch_compute.hpp connects duplicate filtering, vector fetching, and distance computation. The result queue feeds new candidates and thresholds back to the scheduler. Drawing this feedback path and labeling each stream with its record type, count, and termination signal helps establish how the modules coordinate and when a query ends.

The figures below show inter-query and intra-query parallelism. They also include a network interface. The local project used here communicates with the host over PCIe and corresponds to the query processing pipeline and memory channels in the figures.

At each step, graph search uses distances already computed to decide which nodes to visit next. Issuing one candidate and waiting for all its neighbors to finish leaves gaps between memory access and computation. Falcon’s scheduler can issue groups of candidates and keep multiple groups in flight. The multi-DDR host exposes these controls as max_cand_per_group and max_group_num_in_pipe. Increasing them adds work in flight, but also changes expansion order and the amount of search work. Measure execution time, distance computations, and recall together.

The Bloom filter sits before vector fetching and filters candidates that have already been visited. False positives classify some unvisited nodes as visited, so its capacity and hash configuration can affect search quality. Before resizing it, trace state initialization at the start of each query and draining at the end. After a change, measure both the number of filtered requests and recall.

Next, inspect FPGA_multi_DDR/FPGA_intra_query_v1.5_support_batching_longer_FIFO. Its constants.hpp defaults to N_CHANNEL=4 and D=128, with capacities of 64 for the result queue and 32 for the candidate queue. connectivity.cfg requests a 200 MHz kernel clock and maps four sets of vector and adjacency-list ports to four DDR banks.⁹ The build reports provide the achieved timing and resource usage.

The four-channel design shards nodes by ID. In DRAM_utils.hpp, the low CHANNEL_ADDR_BITS bits select the channel, and the shifted ID selects the position within that channel. For four channels, the address calculation is:

1
2
3

channel_id = node_id & 3
local_id   = node_id >> 2
vector_word_offset = local_id * (ceil(D / 16) + 1)

The extra wide word in the formula is a 64-byte visited region for each node. In the export format defined by hnsw.py, the first four bytes hold an int32 marker initialized to −1, followed by 60 zero bytes. With D=128, the vector occupies eight 512-bit words, and the record stride is nine words, or 576 bytes. Vector reads use the valid vector data, while address calculations must use the full stride. Changes to dimensions, channel count, or index format must be reflected consistently in host file loading, kernel address calculation, and the export script.

Intra-query parallelism distributes one query’s work across channels, making it useful for studying how quickly an individual search progresses. Inter-query parallelism assigns queries to parallel processing paths, making it useful for studying throughput under a sustained input stream. Compare the two with a fixed query set, recall target, and request load, and report the batch size and concurrency scheme of each.

Preparing Data and Running Falcon

Download the source and enter the intra-query project:

1
2
3
4

git clone https://github.com/fpgasystems/Falcon-accelerate-graph-vector-search.git
cd Falcon-accelerate-graph-vector-search
git switch --detach 34bcc0aabd04939dce72fd3c257febcf5b8089d2
cd FPGA_multi_DDR/FPGA_intra_query_v1.5_support_batching_longer_FIFO

This project’s host uses OpenCL, and its Makefile links xilinxopencl. The vector addition example uses the XRT native API; Falcon can retain its existing host interface. Reproduction requires adjusting data paths, device selection, and timing boundaries in the host. In the SIFT10M intra-query build, enabling performance monitors caused timing closure to fail. Removing [profile] data=all:all:all from the link configuration allowed the build to complete. Performance monitors consume hardware resources, so recheck implementation timing after changing the monitoring configuration.

The repository’s Python example starts with SIFT1M, matching the current D=128 setting. vector_search_baselines/scripts_hnsw/README_hnsw.md describes HNSW construction and index conversion.¹⁰ The build script depends on Python hnswlib, NumPy, and the repository’s data readers, and contains paths from the authors’ machine. Prepare the base vectors, queries, and ground truth, then update the data directories. Record dependency versions and index construction parameters.

To follow the repository’s example, enter vector_search_baselines/scripts_hnsw and build and convert the index:

1
2
3
4
5
6
7
8

python construct_and_search_hnsw.py \
  --dbname SIFT1M --ef_construction 128 --MD 64 \
  --hnsw_path ../data/CPU_hnsw_indexes

python hnsw_to_FPGA.py \
  --dbname SIFT1M \
  --CPU_index_path ../data/CPU_hnsw_indexes/SIFT1M_index_MD64.bin \
  --FPGA_index_path ../data/FPGA_hnsw/SIFT1M_MD64

For my reproduction experiments, I used a separate preparation workflow: a C++ program based on hnswlib v0.8.0 read big-ann-benchmarks data and built the graph with M=32 and ef_construction=128, then converted the index to Falcon’s FPGA format. M=32 corresponds to a maximum degree of 64 in the bottom HNSW layer. On a small graph of 20,000 nodes, the converter’s output matched the repository’s original conversion script byte for byte. I also searched the original graph with hnswlib to check recall.

The conversion script calls save_as_FPGA_format(..., num_channels=[1, 2, 4]) to generate files for different channel counts. The four-channel host reads ground_vectors_4_chan_0.bin through ground_vectors_4_chan_3.bin and the corresponding ground_links_4_chan_*.bin files, together with metadata and label mappings. Queries also need to be loaded and converted into the format expected by the code. As an initial check of an export, inspect a few nodes: choose a physical ID, read its vector using the channel and local address, check dimensions, values, and neighbor ID ranges, and use the label mapping to identify the original dataset object.

Falcon’s src/host.cpp also contains the authors’ paths for indexes, queries, and ground truth; replace them with your own. Before running, check file existence and sizes, and verify that operations such as fopen and fread succeed. If reading fails, inspect the files and paths first. For HNSW, output IDs pass through labels_base before recall is computed. Recheck this mapping whenever the physical ordering of the graph changes.

With paths and configuration in place, return to the multi-DDR project directory to build. The Makefile lists only src/vadd.cpp as a kernel dependency, so a normal incremental build may reuse an old .xo after changes to headers such as constants.hpp. The -B option forces a rebuild so the updated headers and connectivity take effect.

1

make -B all TARGET=hw PLATFORM="$PLATFORM" > build-hw.log 2>&1

During development, you can set TARGET to hw_emu. The host processes 10000 queries by default. For a short emulation run, adjust the query count, related array sizes, and validation loops together. The final command-line argument, batch_size, controls the query batch size; the host still processes the configured total number of queries.

After building the hardware and checking the data files, run the following when the card is idle:

1

./host xclbin/vadd.hw.xclbin 1 4 64 HNSW SIFT1M 64 10000

The seven arguments after the .xclbin path have the following meanings, in order:

• 1 4: one candidate per group, with up to four groups in flight.
• 64: search parameter ef=64, matching the current result queue capacity.
• HNSW SIFT1M: HNSW graph type and SIFT1M dataset.
• 64 10000: maximum graph degree of 64 and query batch size of 10000.

Before increasing ef, check the queue capacity set at synthesis time. Changes to D, N_CHANNEL, or queue capacity require rebuilding the hardware. When changing the channel count, also select the corresponding connectivity configuration and check that the matching shard files exist.

On the first run, check recall, output ID ranges, and distance values. Keep the index and queries fixed while sweeping candidates per group and groups in flight, for example (1,1), (1,4), and (2,4). Check each setting against hardware queue capacities and scheduler limits. Record Recall@10, device-side time, host-side time, and any available search-work counters for every configuration. If greater concurrency increases distance computations, account for that extra work when explaining timing changes.

The first issue I encountered was input transfer being included in the reported execution time. The original host calls enqueueMigrateMemObjects to upload inputs asynchronously and starts timing immediately afterward. Kernel execution and result transfer are queued behind the upload, and the final q.finish() waits for the entire queue. The printed Duration (including memcpy out) therefore includes any input transfer that is still pending.

Calling q.finish() before starting the timer waits for the upload to complete and separates index loading from query execution. The corrected host timing includes one kernel submission, execution, transfer of output and debug buffers, and host waiting. Device-only kernel time comes from a profiling event or analysis report. A control run with xrt.ini disabled still included input transfer in the host timing, confirming that the timing boundary needed to change.

Falcon’s performance scripts read kernel Total Time from the profiler report. The authors’ test scripts and saved results in perf_test_scripts/saved_df use this timing scope. The correction above applies to the Duration printed by the host and does not change the kernel time used by those scripts.¹¹

The repository’s perf_test_throughput.py reads the kernel’s Total Time (ms) from summary.csv, stores it as time_ms_kernel, and uses it to calculate performance. Before using the script, check which report row utils.py reads and its units, and preserve the original report. The script also uses fixed output filenames, calls DataFrame.append, and ends with rm *csv. Give each experiment its own directory, change the cleanup logic to retain raw data, and check the pandas version or migrate the append operation to the current API. The repository README explicitly notes that experiments sharing these output files cannot run concurrently.¹¹

One Falcon kernel invocation processes the full configured query set, while batch_size controls batches within the kernel. Even with batch_size set to 1, the host’s reported time per query is total time divided by query count, amortizing the submission and result-transfer overhead across queries. To measure individual request latency end to end, record submission and completion times for each request.

Designing Research Experiments

After the first Falcon run, organize experiments around one variable, such as one, two, or four DDR channels, or the effect of candidate group size on latency and search work. For every hardware configuration, save the source commit, patch, build command, platform information, .xclbin SHA-256, achieved II, implementation timing, and LUT, FF, BRAM, URAM, and DSP usage. These records tie performance measurements to the corresponding hardware design.

Vector search performance comparisons also need to account for recall. Fix the dataset, distance metric, ground-truth definition, and query set, then sweep search parameters to obtain recall–throughput or recall–latency curves. Compare CPU, GPU, and FPGA results at similar recall, recording thread counts, concurrency, batch sizes, precision, and data residency. When comparing different index structures, also report index construction parameters, memory footprint, and build cost.

Select configurations using a dedicated tuning query set, and measure final results on held-out test queries. Keep the following records for each experiment:

• Throughput: completed queries and elapsed time in steady state.
• Latency: response times from each request’s start and completion timestamps, including percentiles and request load.
• Run-to-run variation: repeated measurements, warmup procedure, and card temperature.
• Power: sensor source, sampling interval, and measurement scope, distinguishing card power from whole-system power.

In Falcon, more candidates in flight can hide some memory stalls while also causing the search to expand more nodes. Channel utilization, queue waiting, distance computation counts, and recall help separate these effects. The same approach can explain performance changes when modifying your own operator or index.

Closing Notes

From vector addition to Falcon, development on the U250 brings data layout, memory channels, and query scheduling into close interaction. The vector addition measurements show how bank contention, PCIe transfers, and startup overhead affect execution time. Falcon carries these issues into a graph search pipeline with feedback, where recall and search work also shape the results. When bringing your own algorithm to an FPGA, start with a small implementation whose outputs can be checked and whose execution stages can be timed. Use controlled comparisons to locate bottlenecks and guide the organization of parallel computation and data access.