Rubin DP1
Basic characterization of the release catalogs
Last verified run: 2026-09-04
This notebook provides lightweight statistics and diagnostic plots for quick characterization of the release catalogs.
Distributed execution¶
Create the configured Dask cluster for lazy QA operations.
<Client: 'tcp://10.148.0.57:35395' processes=19 threads=38, memory=566.20 GiB> SLURMCluster(3e78b278, 'tcp://10.148.0.57:35395', workers=20, threads=40, memory=596.00 GiB)
Processing catalog 1: object
object¶
Basic Product Information¶
Catalog type: Parquet Parquet files: 29 Catalog size: 6.3 GiB
Total rows: 2,299,757
Total columns: 1,279
Survey Area¶
Computing exact occupied HEALPix pixels from 'coord_ra' and 'coord_dec' at order 12. Dask global deduplication split_out: 64 partitions.
| RA column | Dec column | Pixel source | HEALPix order | NSIDE | Occupied pixels | Pixel area (deg²) | Survey area (deg²) | Total rows | Mean density (objects/deg²) |
|---|---|---|---|---|---|---|---|---|---|
| coord_ra | coord_dec | coord_ra, coord_dec | 12 | 4096 | 47,621 | 0.00020491 | 9.7578 | 2,299,757 | 235,683.6479 |
Unique Count¶
Computing exact global unique count for column 'tract'. Driver collection cap: 10,000 unique values.
Exact global unique count for column 'tract': 25
Unique values for tract
Basic Statistics for Selected Columns¶
| objectId | coord_ra | coord_dec | refExtendedness | refSizeExtendedness | u_cModelFlux | u_cModelFluxErr | g_cModelFlux | g_cModelFluxErr | r_cModelFlux | r_cModelFluxErr | i_cModelFlux | i_cModelFluxErr | z_cModelFlux | z_cModelFluxErr | y_cModelFlux | y_cModelFluxErr | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| count | 2.299757e+06 | 2.299757e+06 | 2.299757e+06 | 1.986580e+06 | 1.839117e+06 | 1.136410e+06 | 1.134564e+06 | 2.047668e+06 | 2.046392e+06 | 2.097704e+06 | 2.096545e+06 | 2.003120e+06 | 2.001897e+06 | 1.702778e+06 | 1.700757e+06 | 1.095722e+06 | 1.092873e+06 |
| mean | -3.575956e+12 | 5.528155e+01 | -2.494006e+01 | 6.862125e-01 | 6.054703e-01 | 5.227689e+03 | 2.639460e+02 | 4.664547e+03 | 7.130965e+01 | 6.723031e+03 | 8.292197e+01 | 7.723372e+03 | 1.260134e+02 | 1.544856e+04 | 2.742437e+02 | 1.664304e+04 | 8.933819e+02 |
| std | 2.098273e+16 | 2.443221e+01 | 2.165932e+01 | 4.640313e-01 | 4.069129e-01 | 1.428670e+05 | 3.120198e+02 | 5.923186e+04 | 1.025800e+02 | 6.872638e+04 | 1.452369e+02 | 7.380798e+04 | 1.580778e+02 | 1.488897e+05 | 3.744301e+02 | 1.842479e+05 | 1.178123e+03 |
| min | 5.795745e+17 | 3.960542e+00 | -7.274139e+01 | 0.000000e+00 | 8.038015e-14 | -4.411273e+05 | 2.112150e+01 | -1.770943e+05 | -1.983902e+01 | -4.339550e+05 | 3.812867e+00 | -1.533093e+04 | 3.355356e+00 | -1.181950e+05 | 1.584127e+01 | -3.487625e+05 | 1.398678e+02 |
| 25% | 6.040621e+17 | 3.823670e+01 | -3.488204e+01 | 0.000000e+00 | 3.670189e-01 | 8.669337e+01 | 1.064152e+02 | 1.476522e+02 | 3.677757e+01 | 2.334023e+02 | 2.887027e+01 | 3.052366e+02 | 6.816216e+01 | 3.278776e+02 | 1.015091e+02 | 5.910406e+02 | 3.830342e+02 |
| 50% | 6.112551e+17 | 5.300007e+01 | -2.795110e+01 | 1.000000e+00 | 9.377733e-01 | 4.120456e+02 | 2.154250e+02 | 3.597480e+02 | 5.325413e+01 | 5.098199e+02 | 6.862202e+01 | 8.798926e+02 | 9.400362e+01 | 8.960273e+02 | 2.246119e+02 | 1.692054e+03 | 7.648067e+02 |
| 75% | 6.287612e+17 | 5.997404e+01 | -1.088257e+01 | 1.000000e+00 | 9.999998e-01 | 1.099977e+03 | 4.396284e+02 | 2.250724e+03 | 1.889290e+02 | 3.450973e+03 | 1.584767e+02 | 4.719375e+03 | 2.554231e+02 | 8.759088e+03 | 4.328556e+02 | 6.562411e+03 | 1.930948e+03 |
| max | 6.500269e+17 | 1.069287e+02 | 8.003709e+00 | 1.000000e+00 | 1.000000e+00 | 2.137514e+07 | 2.517834e+04 | 1.244809e+07 | 1.225165e+04 | 2.473174e+07 | 1.916327e+04 | 2.417323e+07 | 1.129983e+04 | 4.359823e+07 | 2.168403e+04 | 5.201085e+07 | 8.095862e+04 |
Spatial Distribution¶
Magnitude Histogram¶
Magnitude Statistics¶
Statistics include only finite magnitude values in the range -2 ≤ magnitude ≤ 35. Flux columns were converted to magnitudes with magnitude = 31.4 - 2.5 log10(flux). Non-positive, missing, and non-finite flux values are converted to NaN and excluded from finite-value histograms and statistics. Mean, median, P16, P84, and P95 are calculated directly from the filtered values. Peak-bin center is the center of the most populated 0.1-mag bin and should be interpreted as an empirical distribution turnover, not as a calibrated completeness limit. Non-finite, below-range, and above-range values are reported separately and excluded from the descriptive statistics.
| Band | Model | Count | Mean | P16 | Median | P84 | P95 | Peak-bin center | Non-finite | Below range | Above range |
|---|---|---|---|---|---|---|---|---|---|---|---|
| u | PSF | 984,484 | 25.1362 | 24.1333 | 25.3119 | 26.9148 | 28.4249 | 24.9500 | 1,314,653 | 0 | 620 |
| u | Kron | 854,557 | 23.3384 | 22.7617 | 23.8697 | 25.0003 | 25.9382 | 23.4500 | 1,445,119 | 0 | 81 |
| u | cModel | 954,179 | 24.8905 | 23.8787 | 25.1092 | 26.6922 | 28.0830 | 24.7500 | 1,342,272 | 0 | 3,306 |
| u | GAaP 1.0 | 981,471 | 25.0698 | 24.0861 | 25.2545 | 26.8201 | 28.2296 | 24.9500 | 1,317,710 | 0 | 576 |
| g | PSF | 2,000,754 | 25.4147 | 24.3438 | 25.8630 | 27.3648 | 29.2705 | 25.4500 | 298,700 | 0 | 303 |
| g | Kron | 1,912,512 | 24.0032 | 23.1476 | 24.5068 | 25.9250 | 26.7423 | 24.2500 | 387,218 | 0 | 27 |
| g | cModel | 1,947,792 | 25.1572 | 24.0268 | 25.6896 | 27.1786 | 28.9877 | 25.2500 | 350,094 | 0 | 1,871 |
| g | GAaP 1.0 | 2,009,091 | 25.2456 | 24.2069 | 25.7386 | 27.0838 | 28.7329 | 25.3500 | 290,497 | 0 | 169 |
| r | PSF | 2,075,381 | 24.8810 | 23.7732 | 25.3154 | 27.0858 | 28.9273 | 25.1500 | 224,166 | 0 | 210 |
| r | Kron | 2,007,183 | 23.6341 | 22.8432 | 24.1331 | 25.4450 | 26.5281 | 23.9500 | 292,552 | 0 | 22 |
| r | cModel | 2,018,370 | 24.6143 | 23.5520 | 25.1302 | 26.8513 | 28.6263 | 24.8500 | 280,020 | 0 | 1,367 |
| r | GAaP 1.0 | 2,084,062 | 24.6955 | 23.7004 | 25.2119 | 26.7327 | 28.3508 | 24.9500 | 215,613 | 0 | 82 |
| i | PSF | 1,962,700 | 24.5875 | 23.3800 | 24.7197 | 26.8057 | 28.5309 | 24.7500 | 336,842 | 0 | 215 |
| i | Kron | 1,858,749 | 23.4405 | 22.4661 | 23.8077 | 25.4016 | 26.2950 | 23.7500 | 440,970 | 0 | 38 |
| i | cModel | 1,918,314 | 24.2960 | 23.1049 | 24.4780 | 26.5160 | 28.2143 | 24.4500 | 380,709 | 0 | 734 |
| i | GAaP 1.0 | 1,972,629 | 24.4012 | 23.2484 | 24.6132 | 26.4447 | 27.9740 | 24.6500 | 327,052 | 0 | 76 |
| z | PSF | 1,602,816 | 24.2959 | 23.3049 | 25.1243 | 26.5772 | 28.1636 | 24.7500 | 696,750 | 0 | 191 |
| z | Kron | 1,440,574 | 22.9334 | 21.8810 | 23.6449 | 25.0390 | 25.9919 | 23.2500 | 859,144 | 0 | 39 |
| z | cModel | 1,562,992 | 24.0251 | 22.9111 | 24.8577 | 26.3193 | 27.8672 | 24.5500 | 735,442 | 0 | 1,323 |
| z | GAaP 1.0 | 1,604,828 | 24.1441 | 23.1385 | 24.9599 | 26.4002 | 27.7995 | 24.5500 | 694,838 | 0 | 91 |
| y | PSF | 933,068 | 23.8787 | 22.7893 | 24.2920 | 25.8028 | 27.2483 | 24.1500 | 1,366,638 | 0 | 51 |
| y | Kron | 768,613 | 22.1187 | 21.1889 | 22.5477 | 23.7605 | 24.8441 | 22.3500 | 1,531,100 | 0 | 44 |
| y | cModel | 915,763 | 23.5629 | 22.3758 | 24.0399 | 25.5713 | 26.9138 | 24.0500 | 1,382,986 | 0 | 1,008 |
| y | GAaP 1.0 | 925,425 | 23.6091 | 22.5787 | 24.0805 | 25.4955 | 26.8376 | 23.9500 | 1,374,297 | 0 | 35 |
Magnitude Error Histogram¶
Magnitude Error Statistics¶
Statistics include only finite magnitude-error values in the range 0 ≤ magnitude error ≤ 8. Flux-error columns were converted to magnitude errors with sigma_mag = 2.5 / ln(10) * flux_err / flux. Non-positive, missing, and non-finite flux or flux-error values are converted to NaN and excluded from finite-value histograms and statistics. Mean, median, P16, P84, and P95 are calculated directly from the filtered values. Peak-bin center is the center of the most populated 0.01-mag bin. Threshold fractions use Count as their denominator. Non-finite, below-range, and above-range values are reported separately and excluded from the descriptive statistics.
| Band | Model | Count | Mean | P16 | Median | P84 | P95 | Peak-bin center | Fraction ≤ 0.1 | Fraction ≤ 0.2 | Non-finite | Below range | Above range |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| u | PSF | 936,193 | 0.8784 | 0.2530 | 0.5864 | 1.9813 | 4.4558 | 0.2450 | 5.68% | 14.56% | 1,314,653 | 0 | 48,911 |
| u | Kron | 846,462 | 0.5780 | 0.2548 | 0.5197 | 1.1869 | 2.5271 | 0.2950 | 6.67% | 17.26% | 1,445,155 | 0 | 8,140 |
| u | cModel | 913,341 | 0.8690 | 0.2530 | 0.5916 | 1.9696 | 4.3050 | 0.2450 | 5.48% | 14.99% | 1,342,306 | 0 | 44,110 |
| u | GAaP 1.0 | 937,877 | 0.8824 | 0.2565 | 0.5858 | 2.0039 | 4.3715 | 0.2550 | 5.77% | 14.62% | 1,317,710 | 0 | 44,170 |
| g | PSF | 1,967,239 | 0.4348 | 0.0865 | 0.2197 | 0.6718 | 2.0446 | 0.0050 | 31.67% | 55.91% | 298,700 | 0 | 33,818 |
| g | Kron | 1,907,568 | 0.2777 | 0.0778 | 0.2456 | 0.4785 | 0.9936 | 0.0050 | 27.45% | 54.55% | 387,254 | 0 | 4,935 |
| g | cModel | 1,919,648 | 0.4179 | 0.0832 | 0.2123 | 0.6554 | 1.9643 | 0.0050 | 32.60% | 57.13% | 350,109 | 0 | 30,000 |
| g | GAaP 1.0 | 1,989,790 | 0.4093 | 0.0884 | 0.2282 | 0.6612 | 1.7488 | 0.0050 | 30.92% | 54.23% | 290,497 | 0 | 19,470 |
| r | PSF | 2,046,985 | 0.3351 | 0.0631 | 0.1927 | 0.4937 | 1.8762 | 0.0050 | 41.49% | 68.03% | 224,166 | 0 | 28,606 |
| r | Kron | 2,003,169 | 0.2346 | 0.0536 | 0.2117 | 0.4346 | 0.8133 | 0.0050 | 34.43% | 62.15% | 292,635 | 0 | 3,953 |
| r | cModel | 1,995,003 | 0.3226 | 0.0609 | 0.1843 | 0.4879 | 1.8077 | 0.0050 | 42.72% | 69.15% | 280,027 | 0 | 24,727 |
| r | GAaP 1.0 | 2,070,402 | 0.3173 | 0.0649 | 0.1956 | 0.5370 | 1.6545 | 0.0050 | 39.21% | 64.39% | 215,613 | 0 | 13,742 |
| i | PSF | 1,930,582 | 0.3910 | 0.0518 | 0.1767 | 0.6340 | 2.0874 | 0.0050 | 33.79% | 59.96% | 336,842 | 0 | 32,333 |
| i | Kron | 1,852,884 | 0.3182 | 0.0759 | 0.2433 | 0.5197 | 1.0105 | 0.0050 | 25.01% | 47.35% | 440,999 | 0 | 5,874 |
| i | cModel | 1,893,111 | 0.3777 | 0.0507 | 0.1737 | 0.6287 | 2.0409 | 0.0050 | 34.64% | 61.10% | 380,714 | 0 | 25,932 |
| i | GAaP 1.0 | 1,955,413 | 0.3914 | 0.0586 | 0.2095 | 0.7028 | 1.9245 | 0.0050 | 30.99% | 54.19% | 327,052 | 0 | 17,292 |
| z | PSF | 1,559,583 | 0.6035 | 0.1057 | 0.3827 | 1.1593 | 2.8189 | 0.0050 | 26.51% | 41.99% | 696,750 | 0 | 43,424 |
| z | Kron | 1,431,302 | 0.4420 | 0.1140 | 0.3335 | 0.7523 | 1.9192 | 0.0050 | 22.42% | 38.68% | 859,155 | 0 | 9,300 |
| z | cModel | 1,526,157 | 0.5877 | 0.1053 | 0.3714 | 1.1078 | 2.7312 | 0.0050 | 26.64% | 42.51% | 735,467 | 0 | 38,133 |
| z | GAaP 1.0 | 1,572,997 | 0.6101 | 0.1086 | 0.3902 | 1.1626 | 2.7619 | 0.0050 | 25.79% | 40.64% | 694,838 | 0 | 31,922 |
| y | PSF | 884,256 | 1.0262 | 0.1969 | 0.6701 | 2.0477 | 4.3737 | 0.0050 | 12.81% | 21.64% | 1,366,638 | 0 | 48,863 |
| y | Kron | 756,743 | 0.7075 | 0.1959 | 0.5190 | 1.2608 | 2.6630 | 0.0050 | 12.60% | 22.16% | 1,531,112 | 0 | 11,902 |
| y | cModel | 872,536 | 1.0030 | 0.1927 | 0.6594 | 2.0070 | 4.2661 | 0.0050 | 12.93% | 22.07% | 1,382,989 | 0 | 44,232 |
| y | GAaP 1.0 | 886,596 | 1.0208 | 0.1988 | 0.6839 | 2.0860 | 4.2591 | 0.0050 | 12.62% | 21.28% | 1,374,299 | 0 | 38,862 |
Magnitude Error Trends¶
/scripts/luigi.silva/ondemand/automatic-photometric-qa/lib/python3.11/site-packages/dask/dataframe/dask_expr/_concat.py:154: UserWarning: Concatenating dataframes with unknown divisions. We're assuming that the indices of each dataframes are aligned. This assumption is not generally safe. warnings.warn(
Processing catalog 2: ss_object
ss_object¶
Basic Product Information¶
Catalog type: Parquet Parquet files: 1 Catalog size: 293.6 KiB
Total rows: 431
Total columns: 3
Basic Statistics for Selected Columns¶
| ssObjectId | numObs | discoverySubmissionDate | |
|---|---|---|---|
| count | 4.310000e+02 | 431.000000 | 0.0 |
| mean | -2.121543e+16 | 13.893271 | NaN |
| std | 8.246279e+14 | 11.757936 | NaN |
| min | 2.089093e+16 | 1.000000 | NaN |
| 25% | 2.116363e+16 | 6.000000 | NaN |
| 50% | 2.116472e+16 | 11.000000 | NaN |
| 75% | 2.116582e+16 | 19.000000 | NaN |
| max | 2.370033e+16 | 133.000000 | NaN |
Processing catalog 3: ss_source
ss_source¶
Basic Product Information¶
Catalog type: Parquet Parquet files: 1 Catalog size: 1.5 MiB
Total rows: 5,988
Total columns: 23
Basic Statistics for Selected Columns¶
| phaseAngle | heliocentricDist | topocentricDist | heliocentricX | heliocentricY | heliocentricZ | heliocentricVX | heliocentricVY | heliocentricVZ | topocentricX | topocentricY | topocentricZ | topocentricVX | topocentricVY | topocentricVZ | ssObjectId | residualRa | residualDec | diaSourceId | galacticL | galacticB | eclipticLambda | eclipticBeta | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| count | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 | 5.988000e+03 | 5988.000000 | 5988.000000 | 5.988000e+03 | 5988.000000 | 5988.000000 | 5988.000000 | 5988.000000 |
| mean | 12.427374 | 2.537066 | 1.649604 | 1.622136 | 1.854718 | 0.478489 | -0.008154 | 0.006571 | 0.003040 | 1.236830 | 1.024762 | 0.118746 | 0.008022 | 0.000276 | 0.000391 | 2.744868e+14 | -0.000002 | -0.000004 | -3.128205e+14 | 169.061785 | -48.067205 | 39.249311 | -11.720005 |
| std | 4.500287 | 0.443865 | 0.440127 | 0.439058 | 0.272561 | 0.257735 | 0.001719 | 0.001368 | 0.001516 | 0.425688 | 0.269521 | 0.259011 | 0.001690 | 0.001544 | 0.001525 | 9.474196e+14 | 0.000053 | 0.000029 | 2.029209e+13 | 19.940234 | 5.755183 | 8.750900 | 11.828104 |
| min | 6.713130 | 1.138259 | 0.168573 | -0.527252 | 0.922773 | -0.595593 | -0.013355 | -0.003656 | -0.007359 | -0.845027 | 0.102603 | -0.956598 | 0.002758 | -0.008112 | -0.010217 | 2.089093e+16 | -0.000276 | -0.000262 | 6.003334e+17 | 160.518437 | -55.031843 | 36.854435 | -48.806205 |
| 25% | 9.534535 | 2.186638 | 1.283122 | 1.382152 | 1.661247 | 0.507643 | -0.009281 | 0.006024 | 0.002205 | 0.985344 | 0.830853 | 0.150255 | 0.006881 | -0.000656 | -0.000524 | 2.116364e+16 | -0.000013 | -0.000014 | 6.003993e+17 | 161.513196 | -48.595764 | 37.541667 | -7.919423 |
| 50% | 11.075968 | 2.584635 | 1.686183 | 1.674282 | 1.857745 | 0.556021 | -0.007902 | 0.006623 | 0.003074 | 1.292308 | 1.033253 | 0.197827 | 0.008212 | 0.000178 | 0.000392 | 2.116473e+16 | -0.000002 | -0.000004 | 6.004037e+17 | 162.019385 | -48.200297 | 37.863281 | -7.587938 |
| 75% | 13.499421 | 2.829160 | 1.945258 | 1.909028 | 2.034136 | 0.598757 | -0.006920 | 0.007237 | 0.003872 | 1.514523 | 1.200882 | 0.240262 | 0.009225 | 0.001215 | 0.001312 | 2.313393e+16 | 0.000007 | 0.000005 | 6.004081e+17 | 162.566744 | -47.853105 | 38.229448 | -7.246078 |
| max | 36.054829 | 5.274451 | 4.473327 | 3.743195 | 3.778522 | 0.901963 | -0.003902 | 0.010225 | 0.009120 | 3.507115 | 2.922327 | 0.521613 | 0.012436 | 0.004436 | 0.006440 | 2.370033e+16 | 0.000279 | 0.000207 | 6.004654e+17 | 232.858338 | -1.505556 | 109.571784 | -6.596596 |