Roofline’s SDK is accessible through a simple Python interface. Importing models and selecting the target hardware is literally a line of code.
Yes, we support all relevant quantization techniques, such as INT8, blocked, symmetric, asymmetric, and dynamic quantization. Please note, we do not quantize models ourselves. We focus on the deployment infrastructure required to support a broad range of quantization methods from different quantizers.
Effective handling of dynamic shapes is a key strength of our SDK. We combine a set of techniques, e.g., input padding and constrained input shapes, to provide a high-performing and flexible LLM pipeline.
Just reach out through our Contact Us form. We will get in touch immediately.
