util/quantization.cpp¶
Namespaces¶
| Name |
|---|
| sgns |
| sgns::sgprocmanagerquant |
Functions¶
| Name | |
|---|---|
| float | ResolveQuantScale(const std::vector< sgns::Parameter > * parameters) |
| int | ResolveByteQuantMode(const std::vector< sgns::Parameter > * parameters) |
| MNNForwardType | ResolveMnnBackend(const std::vector< sgns::Parameter > * parameters) |
| void | QuantizeFloatBuffer(float * data, size_t count, float scale) |
| void | QuantizeByteBuffer(uint8_t * data, size_t count, int maskBits) |
Functions Documentation¶
function ResolveQuantScale¶
Parameters:
- parameters Job schema's generic parameters array, or nullptr.
Return: The validated, schema-declared scale, or 32768.0f on any invalid/missing declaration.
Phase 14 (QUANT-CFG-01/02, D-01/D-02/D-04/D-05): resolves a job schema-declared "quantScale" entry from the generic parameters array, mirroring the existing find-by-name-in-parameters convention (ParseLayout / ResolveUniforms).
Falls back to the exact v2.1 constant 32768.0f (2^15) – no warning logged, no job rejection – when parameters is null, no entry named "quantScale" of type FLOAT exists, its declared default value is not a JSON number, or the numeric value is not a strictly positive power of two (D-05's mandatory validation, guaranteeing the exact float round-trip property D-03's round(x*S)/S formula relies on can never be silently violated by a bad schema value).
function ResolveByteQuantMode¶
Parameters:
- parameters Job schema's generic parameters array, or nullptr.
Return: The validated, schema-declared mask-bit count in [0, 8], or 0 on any invalid/missing declaration.
Phase 14 (QUANT-CFG-01/02, D-02/D-07/D-08): resolves a job schema-declared "byteQuantMode" entry from the generic parameters array, same lookup convention as ResolveQuantScale.
Falls back to 0 (the exact v2.1 byte-identity no-op) – no warning, no job rejection – when parameters is null, no entry named "byteQuantMode" of type INT exists, its declared default value is not a JSON integer, or the integer value falls outside the inclusive range [0, 8]. N=8 (masking all 8 bits) is a valid, non-fallback boundary value by design (D-07/D-08); N=9 and above fall back to 0.
function ResolveMnnBackend¶
Parameters:
- parameters Job schema's generic parameters array, or nullptr.
Return: MNN_FORWARD_CPU only for an explicit "cpu" declaration; MNN_FORWARD_VULKAN for "vulkan" and every fallback case.
Phase 13 (SGF-01, D-04/D-05): resolves a job schema-declared "backend" entry from the generic parameters array, same lookup convention as ResolveQuantScale/ResolveByteQuantMode, selecting the MNN session backend for MNN-based processors.
Falls back to MNN_FORWARD_VULKAN – the exact behavior every MNN processor had when config.type was hardcoded – when parameters is null, no entry named "backend" of type STRING exists, its declared default value is not a JSON string, or the lowercased string is neither "cpu" nor "vulkan" (T-13-02: an untrusted schema value can never select an unintended backend; it only ever falls back to the safe default).
function QuantizeFloatBuffer¶
Parameters:
- data Pointer to a float buffer to quantize in place.
- count Number of float elements in the buffer.
- scale The resolved scale S to use (see ResolveQuantScale()).
Phase 12 real implementation (D-03 through D-09): IEEE-754 special-value canonicalization followed by fixed-precision scale-round-cast quantization.
Canonicalization (evaluated strictly before any rounding arithmetic, D-07):
- Denormals (both signs) flush to canonical +0.0 (0x00000000), D-07/D-08.
- NaN (any payload/sign/signaling bit) canonicalizes to the hardcoded quiet-NaN bit pattern 0x7FC00000 (D-09) – never std::numeric_limits
::quiet_NaN(), since that is not guaranteed to be bit-identical across compilers/platforms. - +Inf / -Inf canonicalize to two distinct fixed bit patterns, 0x7F800000 / 0xFF800000 respectively (D-06) – never collapsed to one value, so a wrong-sign divergence stays visible to SECV-01's counter-test.
- -0.0 and +0.0 both collapse to the single canonical zero bit pattern 0x00000000, sign discarded (D-08).
Rounding (only reached once every canonicalization branch above has been evaluated and found not to apply): q = round(x * S) / S, with S = 2^15 (32768.0f) – a power-of-two scale factor for exact float round-tripping. The tolerance is a single fixed absolute epsilon (D-04) – not magnitude-adaptive, not relative/ULP-based, and not schema-configurable.
Original Phase 12 derivation (D-05): S = 2^20 (1048576.0f), grid step ~9.5367431640625e-07, chosen for a ~9.14x margin over Phase 11's measured cross-machine (Mac vs Windows) MNN float32 divergence: maxAbsDelta ≈ 1.043081283569336e-07, maxRelDelta ≈ 7.269731577252969e-05, maxUlpDistance = 768 (512-element float32 MNN fixture; see 11-CAPTURE-RESULTS.md).
Phase 13 Plan 13-04 gap-closure revision (this constant's current value): Phase 13's own fresh re-validation (13-SCOPE-BOUNDARY.md) measured a post-quantization maxAbsDelta of exactly 9.5367431640625e-07 – one full old-grid step – with 12 of 15 MNN chunk hashes still diverging cross-hardware at the old S=2^20 grid, direct evidence the original ~9x margin was insufficient against per-element grid-boundary tie-break divergence for this fixture's real data.
A local binary search over power-of-two S values (13-04-PLAN.md Task 1, revised approach) against processing_conformance_security_test's Secv01CounterTest.MnnCorruptedModelStillDiverges bracketed a hard boundary: S=2^15 (grid step 3.0517578125e-05) passes – the corrupted MNN model's artifactId still diverges from the correct model's, as SECV-01 requires – while S=2^14 (grid step 6.103515625e-05) FAILS deterministically (the corrupted model's post-quantization artifactId collides bit-for-bit with the correct model's, confirmed by re-running twice, not flaky). S=2^15 was chosen over S=2^14 specifically to keep one full power-of-two step of margin above this confirmed failure boundary rather than sitting at the exact edge (floating-point behavior can vary subtly build-to-build). S=2^15's grid step is 32x the original S=2^20 grid step and ~292x Phase 11's originally-measured maxAbsDelta – substantially reducing (not mathematically eliminating) the per-element grid-boundary tie-break collision probability for this fixture's real values, while every SECV-01 case (corrupted MNN model, wrong render shader constant) still passes.
This is a probabilistic engineering mitigation, not a one-shot guaranteed solution: a fixed rounding grid cannot mathematically guarantee zero cross-hardware divergence for arbitrary per-element deltas that happen to land arbitrarily close to a rounding boundary – it only reduces the probability of that happening for this fixture's actual values. See 13-SCOPE-BOUNDARY.md's Refit section (Plan 13-05) for the fresh empirical cross-machine outcome this constant change is validated against.
Phase 14 (QUANT-CFG-01/02): S is now schema-configurable via the caller-resolved scale argument, produced by calling ResolveQuantScale() once per StartProcessing() invocation. 32768.0f remains the exact fallback when nothing valid is schema-declared, and the D-03 round(x*S)/S formula plus the D-06..D-09 canonicalization branch order above are entirely unchanged by this addition – this paragraph documents schema-configurability, it does not revise or contradict the S=2^15 derivation history above it.
function QuantizeByteBuffer¶
Parameters:
- data Pointer to a byte buffer to quantize in place.
- count Number of bytes in the buffer.
- maskBits Number of low bits to clear per byte, in [0, 8] (see ResolveByteQuantMode()); <= 0 is the identity no-op.
Phase 12 deliberate identity pass-through for the render uint8 path.
This is a considered design decision for this phase, not an inherited Phase 10 placeholder: Phase 11's empirical render fixture data (256-element uint8 RGBA/RGB pixel output, Mac vs Windows) showed contentHashMatch: true with maxAbsDelta/maxRelDelta/maxUlpDistance all 0.0 – no observed cross-hardware divergence in the uint8 render path this milestone's fixtures exercise (see 11-CAPTURE-RESULTS.md). Applying a lossy tolerance-band here with no empirical justification would only enlarge the space of results indistinguishable from a correct one, so this stays byte-identity until new fixture data shows otherwise.
Phase 14 (QUANT-CFG-01/02, D-06/D-07): the mask is now schema- configurable via the caller-resolved maskBits argument, produced by calling ResolveByteQuantMode() once per StartProcessing() invocation. maskBits <= 0 (D-07's N=0/absent case) remains the exact v2.1 byte-identity no-op; otherwise the low maskBits bits of every byte are cleared (D-06's bit-masking technique, value &= ~((1<<N)-1)).
Source code¶
#include "util/quantization.hpp"
#include <algorithm>
#include <cctype>
#include <cmath>
#include <cstring>
namespace sgns::sgprocmanagerquant
{
namespace
{
// Phase 14 D-05/Pitfall 2: never use std::log2/std::pow here -- a
// transcendental-function-based check's last-bit behavior is
// platform-dependent, which would reintroduce exactly the
// cross-hardware nondeterminism this milestone exists to eliminate.
// The integer bit-trick below is deterministic on every platform.
bool IsPositivePowerOfTwo( double value )
{
if ( !( value > 0.0 ) )
{
return false;
}
if ( std::floor( value ) != value )
{
return false;
}
const auto asInt = static_cast<uint64_t>( value );
return asInt != 0u && ( asInt & ( asInt - 1u ) ) == 0u;
}
} // namespace
float ResolveQuantScale( const std::vector<sgns::Parameter> *parameters )
{
constexpr float kFallbackScale = 32768.0f; // 2^15, exact v2.1 constant (D-04)
if ( parameters )
{
for ( const auto ¶m : *parameters )
{
if ( param.get_name() == "quantScale" && param.get_type() == sgns::ParameterType::FLOAT )
{
const auto &def = param.get_parameter_default();
if ( def.is_number() )
{
const double declared = def.get<double>();
if ( IsPositivePowerOfTwo( declared ) )
{
return static_cast<float>( declared );
}
}
break;
}
}
}
return kFallbackScale;
}
int ResolveByteQuantMode( const std::vector<sgns::Parameter> *parameters )
{
constexpr int kFallbackMaskBits = 0; // Identity no-op, exact v2.1 behavior (D-08)
if ( parameters )
{
for ( const auto ¶m : *parameters )
{
if ( param.get_name() == "byteQuantMode" && param.get_type() == sgns::ParameterType::INT )
{
const auto &def = param.get_parameter_default();
if ( def.is_number_integer() )
{
const int declared = def.get<int>();
if ( declared >= 0 && declared <= 8 )
{
return declared;
}
}
break;
}
}
}
return kFallbackMaskBits;
}
MNNForwardType ResolveMnnBackend( const std::vector<sgns::Parameter> *parameters )
{
// Phase 13 (D-04): MNN_FORWARD_VULKAN is the fallback so every
// existing caller (no "backend" parameter declared) keeps today's
// exact hardcoded-Vulkan behavior.
constexpr MNNForwardType kFallbackBackend = MNN_FORWARD_VULKAN;
if ( parameters )
{
for ( const auto ¶m : *parameters )
{
if ( param.get_name() == "backend" && param.get_type() == sgns::ParameterType::STRING )
{
const auto &def = param.get_parameter_default();
if ( def.is_string() )
{
// Lowercase-normalize so "CPU"/"Cpu" behave as "cpu"
// (T-13-02 mitigation: normalization happens before
// the accept-list check, and anything outside the
// two accepted values still falls back to Vulkan).
std::string declared = def.get<std::string>();
std::transform( declared.begin(),
declared.end(),
declared.begin(),
[]( unsigned char c ) { return static_cast<char>( std::tolower( c ) ); } );
if ( declared == "cpu" )
{
return MNN_FORWARD_CPU;
}
if ( declared == "vulkan" )
{
return MNN_FORWARD_VULKAN;
}
}
break;
}
}
}
return kFallbackBackend;
}
void QuantizeFloatBuffer( float *data, size_t count, float scale )
{
// Phase 13 Plan 13-04 gap-closure widening (supersedes Phase 12 D-05's
// 2^20 value): the original S=2^20 grid step (9.5367431640625e-07)
// gave only a ~9.14x margin over Phase 11's measured cross-machine
// maxAbsDelta (1.043081283569336e-07); Phase 13's own fresh
// re-validation (13-SCOPE-BOUNDARY.md) measured a post-quantization
// maxAbsDelta of exactly 9.5367431640625e-07 (one full old-grid step)
// with 12 of 15 MNN chunk hashes still diverging cross-hardware --
// direct evidence the ~9x margin was insufficient.
//
// A local binary search over power-of-two S values (13-04-PLAN.md
// Task 1, revised approach) against processing_conformance_security_
// test's Secv01CounterTest.MnnCorruptedModelStillDiverges found:
// S=2^20 (9.5367431640625e-07 grid step) -- SECV-01 passes (baseline)
// S=2^17 (7.62939453125e-06 grid step) -- SECV-01 passes
// S=2^16 (1.52587890625e-05 grid step) -- SECV-01 passes
// S=2^15 (3.0517578125e-05 grid step) -- SECV-01 passes
// S=2^14 (6.103515625e-05 grid step) -- SECV-01 FAILS (the
// deliberately corrupted MNN model's artifactId collides
// bit-for-bit with the correct model's, memcmp equal, 0 vs 0 --
// confirmed deterministic, not flaky, by re-running twice)
// S=2^15 is chosen: the widest power-of-two grid step confirmed safe,
// one full power-of-two step of margin above the confirmed S=2^14
// failure boundary (not the exact edge), giving 32x the old S=2^20
// grid step (~292x Phase 11's original maxAbsDelta) while still
// leaving SECV-01's corrupted-model divergence fully intact.
//
// Phase 14 (QUANT-CFG-01/02): this constant is no longer hardcoded
// here -- callers resolve it via ResolveQuantScale() (D-04/D-05
// fallback to this exact 32768.0f value) and pass it as `scale`.
for ( size_t i = 0; i < count; ++i )
{
float x = data[i];
// Extract the bit pattern via memcpy (never a reinterpret_cast
// type-pun), mirroring HalfToFloat's existing bit-punning style
// (processing_processor_mnn_float.cpp).
uint32_t bits = 0;
std::memcpy( &bits, &x, sizeof( bits ) );
const uint32_t exponentBits = bits & 0x7F800000u;
const uint32_t mantissaBits = bits & 0x007FFFFFu;
// Branch order is itself the D-07 requirement: every special-value
// check below is evaluated before the rounding arithmetic in the
// final else arm ever runs.
// 1. Denormal (both signs, D-07): biased exponent field is zero but
// mantissa is nonzero. Flush to canonical +0.0 (D-08).
if ( exponentBits == 0u && mantissaBits != 0u )
{
data[i] = 0.0f;
}
// 2. NaN: canonicalize to the hardcoded quiet-NaN bit pattern
// 0x7FC00000 (D-09), regardless of payload/sign/signaling bit.
else if ( std::isnan( x ) )
{
constexpr uint32_t kCanonicalNaN = 0x7FC00000u;
std::memcpy( &data[i], &kCanonicalNaN, sizeof( kCanonicalNaN ) );
}
// 3. Infinity: two distinct fixed bit patterns (D-06), never
// collapsed to one value.
else if ( std::isinf( x ) )
{
constexpr uint32_t kPositiveInfinity = 0x7F800000u;
constexpr uint32_t kNegativeInfinity = 0xFF800000u;
if ( std::signbit( x ) )
{
std::memcpy( &data[i], &kNegativeInfinity, sizeof( kNegativeInfinity ) );
}
else
{
std::memcpy( &data[i], &kPositiveInfinity, sizeof( kPositiveInfinity ) );
}
}
// 4. Signed zero (D-08): +0.0 and -0.0 both compare equal to 0.0f
// under IEEE equality; collapse to the single canonical zero.
else if ( x == 0.0f )
{
data[i] = 0.0f;
}
// 5. Ordinary finite value: fixed-point scale-round-cast (D-03).
else
{
data[i] = std::round( x * scale ) / scale;
}
}
}
void QuantizeByteBuffer( uint8_t *data, size_t count, int maskBits )
{
// D-01/QUANT-04: byte-identity no-op when nothing (valid) is
// schema-declared -- see header doc comment for the Phase 11
// empirical justification (contentHashMatch: true, all deltas 0.0).
// Phase 14 D-07: maskBits<=0 (absent/N=0) is exactly this v2.1
// identity behavior, unchanged.
if ( maskBits <= 0 )
{
return;
}
// D-06: clear the low `maskBits` bits of every byte. maskBits is
// resolver-validated to [0, 8] (ResolveByteQuantMode), so the shift
// below never exceeds the width of an unsigned int.
const uint8_t mask = static_cast<uint8_t>( ~( ( 1u << maskBits ) - 1u ) );
for ( size_t i = 0; i < count; ++i )
{
data[i] &= mask;
}
}
} // namespace sgns::sgprocmanagerquant
Updated on 2026-09-17 at 06:29:15 +0000