What’s New

1. What’s New#

This page explains the new features in the current release. This cannot cover all the changes in all packages but should help users to find out what needs to be adapted when changing to the new release.

1.1. Changes since release-09#

1.1.1. Changes in the analysis package in light-2409-toyger#

Modules#

  • Added a module VariabesToTable that writes Ntuples to hdf5, csv or parquet.

  • Added support for the FEI table format of the corrections repository to the analysis reweighting script

Graph Neural Network based Flavor Tagger#

  • Fixed a memory leak that affected light-2406-ragdoll and light-2408-savannah by upgrading the tensorflow version

1.1.2. Changes in the analysis package in light-2408-savannah#

Modules#

  • Fixed a bug in MCDecayFinder, which previously didn’t find all true candidates when using the default arrow.

  • Fixed a bug in Offline reweighting where the weights were not correctly assigned if a selection cut was applied on the input DataFrame.

Variables#

  • Added track variables d0FromIP, z0FromIP and phi0FromIP; which return the tracking parameters of the point-of-closest-approach (POCA), with respect to the measured beam interaction point.

  • Added energy [in tau rest frame] of radiated photon from tau+/- decays: ‘tauPlusEgstar’/’tauMinusEgstar’

  • Added a meta variable useMCancestorBRestFrame which returns the given variable in the rest frame of the ancestor B MC particle.

Command line tools#

Charm Flavor Tagger#

  • Updated the default identifier name for the payload of the Charm Flavor Tagger (argument uniqueIdentifier of the function charmFlavorTagger) to CFT_ragdoll. This value must be used for all releases as recent as light-2406-ragdoll and release-09.

1.2. Changes since release-08#

New and updated packages in externals

Table 1.1 New and updated packages in externals from v02-00-02 to v02-02-04#

package

old version

new version

belle_legacy

1.10

1.11

binutils

2.36.1

2.41

boost

1.78.0

1.85.0

clhep

2.4.6.2

2.4.7.1

cmake

3.22.2

3.28.3

davix

0.7.6

0.8.6

doxygen

1.9.1

1.9.6

gcc

11.2.0

14.1.0

gdb

10.2

14.2

Geant4

11.1.1

11.2.1

Hammer

1.3.0

1.4.1

isl

0.24

0.26

llvm

12.0.0

18.1.6

Millepede-II

V04-10-00

V04-16-03

openssl (only for EL7)

3.0.14

python

3.8.8

3.11.9

ROOT

6.24/06

6.26/14

TBB

2021.9.0

Vc

1.4.3

1.4.5

VecCore

0.8.0

0.8.1

VecGeom

1.2.1

1.2.8

VGM

5.0

5.2

XRootD

5.4.3

5.6.1

absl-py

1.3.0

2.1.0

accessible-pygments

0.0.5

aiohttp

3.8.1

3.9.5

aiosignal

1.2.0

1.3.1

alabaster

0.7.12

0.7.16

anyio

4.4.0

appdirs

1.4.4

removed

argon2-cffi

21.3.0

23.1.0

arrow

1.3.0

asttokens

2.4.1

async-lru

2.0.4

async-timeout

4.0.2

removed

attrs

21.4.0

23.2.0

autocommand

2.2.2

autopep8

1.6.0

2.3.1

awkward

1.7.0

2.6.5

awkward-cpp

34

awkward-pandas

2023.8.0

Babel

2.9.1

2.15.0

backcall

0.2.0

removed

backports.tarfile

1.2.0

beautifulsoup4

4.11.1

4.12.3

bleach

4.1.0

6.1.0

blosc2

2.7.0

boost-histogram

1.2.1

1.4.1

cachetools

4.2.4

5.3.3

certifi

2021.10.8

2024.6.2

cffi

1.15.0

1.16.0

charset-normalizer

2.0.10

3.3.2

cheroot

8.6.0

10.0.1

CherryPy

18.6.1

18.10.0

click

8.1.7

cloudpickle

2.0.0

3.0.0

comm

0.2.2

configargparse

1.5.3

removed

connection-pool

0.0.3

removed

contourpy

1.2.1

cramjam

2.8.3

cryptography

36.0.1

removed

cycler

0.11.0

0.12.1

Cython

0.29.26

3.0.10

datrie

0.8.2

removed

debugpy

1.5.1

1.8.1

dgl

0.6.1

2.1.0

dm-tree

0.1.6

0.1.8

docutils

0.17.1

0.21.2

entrypoints

0.3

removed

eoshep

1.0.1

removed

executing

2.0.1

fastjsonschema

2.20.0

file-read-backwards

2.0.0

3.1.0

filelock

3.4.2

3.15.4

flake8

4.0.1

7.1.0

flatbuffers

22.9.24

24.3.25

fonttools

4.28.5

4.53.0

fqdn

1.5.1

frozenlist

1.3.0

1.4.1

fsspec[http]

2022.5.0

2024.6.0

future

0.18.2

removed

gast

0.3.3

0.5.4

gitdb

4.0.9

4.0.11

GitPython

3.1.26

3.1.43

google-auth

2.3.3

removed

google-auth-oauthlib

0.4.6

removed

grpcio

1.32.0

1.64.1

h11

0.14.0

h5py

2.10.0

3.11.0

httpcore

1.0.5

httpx

0.27.0

idna

3.3

3.7

imagesize

1.3.0

1.4.1

iminuit

2.26.0

importlib-metadata

4.10.0

7.2.0

importlib-resources

5.4.0

removed

inflect

7.3.0

iniconfig

1.1.1

2.0.0

ipykernel

6.6.1

6.29.4

ipython

7.31.0

8.25.0

ipython-genutils

0.2.0

removed

ipywidgets

7.6.5

8.1.3

isoduration

20.11.0

jaraco.classes

3.2.1

removed

jaraco.collections

3.5.1

5.0.1

jaraco.context

5.3.0

jaraco.functools

3.5.0

4.0.1

jaraco.text

3.6.0

3.12.1

jedi

0.18.1

0.19.1

jeepney

0.7.1

removed

Jinja2

3.0.3

3.1.4

jira

3.1.1

3.8.0

joblib

1.1.0

1.4.2

json5

0.9.25

jsonpointer

3.0.0

jsonschema[format-nongpl]

4.3.3

4.22.0

jsonschema-specifications

2023.12.1

jupyter-client

7.1.0

8.6.2

jupyter-console

6.4.0

6.6.3

jupyter-core

4.9.1

5.7.2

jupyter-events

0.10.0

jupyter-lsp

2.2.5

jupyter-server

2.14.1

jupyter-server-terminals

0.5.3

jupyterlab

4.2.2

jupyterlab-pygments

0.1.2

0.3.0

jupyterlab-server

2.27.2

jupyterlab-widgets

1.0.2

3.0.11

jupytext

1.13.6

1.16.2

keras

2.10.0

3.3.3

keras-preprocessing

1.1.2

removed

keyring

23.5.0

removed

kiwisolver

1.3.2

1.4.5

libclang

14.0.6

18.1.1

lightgbm

3.3.2

4.4.0

lightning-utilities

0.11.2

lxml

4.7.1

5.2.2

Markdown

3.3.6

3.6

markdown-it-py

1.1.0

3.0.0

MarkupSafe

2.0.1

2.1.5

matplotlib

3.5.1

3.9.0

matplotlib-inline

0.1.3

0.1.7

mccabe

0.6.1

0.7.0

mdit-py-plugins

0.3.0

0.4.1

mdurl

0.1.2

metakernel

0.28.2

0.30.2

mistune

0.8.4

3.0.2

ml-dtypes

0.3.2

more-itertools

8.12.0

10.3.0

mpmath

1.3.0

msgpack

1.0.8

multidict

6.0.2

6.0.5

mysql-connector-python

8.0.27

8.4.0

namex

0.0.8

nbclient

0.5.9

0.10.0

nbconvert

6.4.0

7.16.4

nbformat

5.1.3

5.10.4

nbsphinx

0.8.8

0.9.4

ndindex

1.8

nest-asyncio

1.5.4

1.6.0

networkx

2.7.1

3.3

notebook

6.4.6

7.2.1

notebook-shim

0.2.4

numexpr

2.8.1

2.10.1

numpy

1.20.1

1.26.4

oauthlib

3.1.1

3.2.2

optree

0.11.0

overrides

7.7.0

packaging

21.3

24.1

pandas

1.3.5

2.2.2

pandocfilters

1.5.0

1.5.1

parso

0.8.3

0.8.4

pexpect

4.8.0

4.9.0

pickleshare

0.7.5

removed

pillow

9.0.0

10.3.0

pip

21.3.1

24.1

platformdirs

4.2.2

plothist

1.2.4

pluggy

1.0.0

1.5.0

portend

3.1.0

3.2.0

prometheus-client

0.12.0

0.20.0

prompt-toolkit

3.0.24

3.0.47

protobuf

3.19.3

4.25.3

psutil

5.9.0

6.0.0

ptyprocess

0.7.0

removed

pulp

2.6.0

removed

pure-eval

0.2.2

py

1.11.0

removed

py-cpuinfo

9.0.0

pyarrow

16.1.0

pyasn1

0.4.8

removed

pyasn1-modules

0.2.8

removed

pycodestyle

2.8.0

2.12.0

pycparser

2.21

2.22

pydata-sphinx-theme

0.8.1

0.15.3

pyDeprecate

0.3.1

removed

pydocstyle

6.1.1

6.3.0

pyflakes

2.4.0

3.2.0

Pygments

2.11.2

2.18.0

pyhepmc

2.13.4

PyJWT

2.8.0

pylama

8.3.7

8.4.1

pyparsing

3.0.6

3.1.2

pypmc

1.2

removed

pyroofit

0.9.6

removed

pyrsistent

0.18.0

removed

pytest

6.2.5

8.2.2

python-dateutil

2.8.2

2.9.0.post0

python-gitlab

3.9.0

4.6.0

python-json-logger

2.0.7

pytorch-ignite

0.5.0.post2

pytorch-lightning

1.5.6

2.3.0

pytz

2021.3

2024.1

PyYAML

6.0

6.0.1

pyzmq

22.3.0

26.0.3

qtconsole

5.2.2

5.5.2

QtPy

2.0.0

2.4.1

ratelimiter

1.2.0.post0

removed

referencing

0.35.1

requests

2.27.1

2.32.3

requests-oauthlib

1.3.0

2.0.0

requests-toolbelt

0.9.1

1.0.0

rfc3339-validator

0.1.4

rfc3986-validator

0.1.1

rich

13.7.1

root-numpy

4.8.0

removed

root-pandas

0.7.0

removed

rpds-py

0.18.1

rsa

4.8

removed

scikit-learn

1.0.2

1.5.0

scipy

1.7.3

1.13.1

SCons

4.3.0

4.7.0

seaborn

0.11.2

0.13.2

SecretStorage

3.3.1

removed

Send2Trash

1.8.0

1.8.3

setuptools

60.5.0

70.1.0

six

1.15.0

1.16.0

smmap

5.0.0

5.0.1

snakemake

6.13.1

removed

sniffio

1.3.1

soupsieve

2.3.2.post1

2.5

Sphinx

4.3.2

7.3.7

sphinx-book-theme

0.3.3

1.1.3

sphinx-codeautolink

0.15.0

0.15.2

sphinxcontrib-applehelp

1.0.2

1.0.8

sphinxcontrib-devhelp

1.0.2

1.0.6

sphinxcontrib-htmlhelp

2.0.0

2.0.5

sphinxcontrib-qthelp

1.0.3

1.0.7

sphinxcontrib-serializinghtml

1.1.5

1.1.10

splinter

0.21.0

stack-data

0.6.3

stopit

1.1.2

removed

sympy

1.12.1

tables

3.7.0

3.9.2

tabulate

0.8.9

0.9.0

tempora

5.0.0

5.6.0

tensorboard

2.10.1

2.17.1

tensorboard-data-server

0.6.1

0.7.2

tensorboard-plugin-wit

1.8.1

removed

tensorflow

2.10.0

2.17.0

tensorflow-estimator

2.10.0

removed

tensorflow-io-gcs-filesystem

0.27.0

0.37.0

tensorflow-probability

0.15.0

0.24.0

termcolor

1.1.0

2.4.0

terminado

0.12.1

0.18.1

testpath

0.5.0

removed

threadpoolctl

3.0.0

3.5.0

tinycss2

1.3.0

toml

0.10.2

removed

toposort

1.7

removed

torch

1.11.0

2.2.2

torch_cluster

1.6.3

torch-geometric

2.0.4

2.5.3

torch_scatter

2.0.9

2.1.2

torch_sparse

0.6.14

0.6.18

torch_spline_conv

1.2.2

torchaudio

0.11.0

2.2.2

torchdata

0.7.1

torchmetrics

0.9.3

1.4.0.post0

torchvision

0.12.0

0.17.2

tornado

6.1

6.4.1

tqdm

4.62.3

4.66.4

traitlets

5.1.1

5.14.3

typeguard

4.3.0

types-python-dateutil

2.9.0.20240316

typing-extensions

3.7.4.3

4.12.2

tzdata

2024.1

uncertainties

3.1.6

3.2.1

uproot

4.1.9

5.3.7

uri-template

1.3.0

urllib3

1.26.8

2.2.2

wcwidth

0.2.5

0.2.13

webcolors

24.6.0

websocket-client

1.8.0

Werkzeug

2.0.2

3.0.3

wheel

0.37.1

0.43.0

widgetsnbextension

3.5.2

4.0.11

wrapt

1.16.0

xgboost

1.6.1

1.7.6

yarl

1.9.4

zc.lockfile

3.0.post1

zipp

3.19.2

zstandard

0.16.0

0.22.0

1.2.1. Changes in the analysis package in light-2406-ragdoll#

Modules#

  • Added the ParticleMassHypothesesUpdater module, which updates the mass hypothesis of a given input particle list. The module takes as input a particle list and a PDG code and creates a new particle list containing copies of the original particles, with updated mass hypotheses. The only supported mass hypotheses are \(e, \mu, K, \pi, p\) (for both input and output lists). This module differs from the ParticleMassUpdater module, since the latter changes the value of the mass of the particles without changing their PDG code.

GraFEI#

  • The tool now assigns an updated mass hypothesis to each final state particle corresponding to the model prediction. This is done by filling new particles lists named :graFEI.

Variables#

  • PIDVariables: in all functions replacing logL == 0 with a call to isAvailable which tests the detector flags in PIDLikelihoods

  • Added variables: isPIDAvailable, isPIDAvailableFrom, pidLogarithmicProbabilityExpert

  • in function addAlias(alias, variable) adding return alias for convenience

  • Added a new Neural-Network based KLM muon probability variable klmMuonIDDNN (for expert use only).

1.2.2. Changes in the analysis package in light-2405-quaxo#

Modules#

Variables#

1.2.3. Changes in the analysis package in light-2403-persian#

Modules#

  • Added a new option storeEventType in VariablesToNtuple and VariablesToEventBasedTree to store the MC event type information in the branch __eventType__ with the string data type, e.g. mixed, ccbar. This information is available from MC16 on. For MC15 or older and for data, the branch will be empty. By default, the option is true.

Command line tools#

Variables#

  • Added variables for the MCMatching of ECLCluster based Klong, clusterTotalMCMatchWeightForKlong, clusterTotalMCMatchWeightForBestKlong, and varForBestMatchedMCKlong.

  • Added variables for MC value of helix (track) parameters: mcD0, mcPhi0, mcOmega, mcZ0, mcTanLambda. These variable work both on MC-matched track-based Particles and on Particles made from MCParticles.

  • Added the variable genQ2PmPd, which calculates the generated value of the four-momentum transfer squared using the four-momentum difference of the mother particle and given daughter particles: \((p_m - p_{d_i} - p_{d_j} - ...)^2\).

  • Added MetaVariable deltaPhiToMostB2BPhiInList, which returns the abs(phi difference) to the most back-to-back in phi (closest to 180 degrees) particle in the provided particle list.

  • Added the variable ancestorBIndex, which returns the array index of the B ancestor of the particle if a MC-match is found and the B exists, and -1 if not.

GraFEI#

  • A first version of the Graph-based Full Event Interpretation (GraFEI) is introduced. The basic principles and instructions on how to train and apply the model are available in the dedicated page.

1.2.4. Changes in the analysis package in light-2401-ocicat#

Variables#

  • The variable daughterDiffOf accepts generalized daughter indices now so that you can calculate the difference of any variable between particles of different generations. The variable grandDaughterDiffOf is therefore not needed anymore and declared deprecated. The variables mcDaughterDiffOf and daughterNormDiffOf also accept generalized daughter indices now.

  • Added the BelleTofMass variable (Belle only), which gives the mass of a track in \(GeV/c^2\), calculated using the TOF information, the path length and the momentum of the particle.

1.2.5. Changes in the analysis package in light-2311-nebelung#

Modules#

  • Added a new option useFloat in VariablesToNtuple and modularAnalysis.variablesToNtuple used to store floating-point data as single-precision (float) numbers instead of double-precision (double) numbers.

  • Fixed a bug of MCMatching in HelixErrorScaler module, which is used for stdV0s.scaleErrorKshorts function. The MCMatching properties of K_S0 can be configured by the decayString in modularAnalysis.fillParticleList. However, the properties were not correctly propagated to the new V0 particles created with HelixErrorScaler. This bug caused the loss of signal (isSignal==1) candidates by about 5% for B-meson decays due to missing radiated photons, which should be accepted.

  • Added a new module TwoBodyISRPhotonCorrector which can be accessed in the new function modularAnalysis.twoBodyISRPhotonCorrector. This module is meant to be used in two-body decays involving a high-energy ISR photon that likely experiences ECL leakage leading to an inaccurate energy measurement. The energy is corrected kinematically based on the beam energy, the photon’s direction, and the mass of the other particle in the two-body decay. The corrected kinematics are stored in copied particles in a new ParticleList. The original kinematics of the ISR photon can be accessed using the metavariable originalParticle.

  • Added a new module TrackingEnergyLossCorrection alongside its helper function modularAnalysis.correctTrackEnergy. This module modifies the track energy of each particle inside a particle list according to the given correction value or by a phase-space dependent correction value extracted from a ParticleWeightingLookUpTable payload. This module is intended to be used alongside the momentum scale factors implemented in the module TrackingMomentumScaleFactors.

  • Fixed a bug in the MCDecayFinder and modularAnalysis.findMCDecay that causes a segmentation fault when multiple candidates are found.

Variables#

  • Fixed a bug for the variable cosHelicityAngleMomentum. The variable always returned 1 when a daughter particle decays to \(e^+ e^-\), e.g. \(B^0 \to J/\psi (\to e^+ e^-) K_S^0\). This is due to a special treatment for the pi0 Dalitz decay, i.e. \(\pi^0 \to \gamma (\to e^+ e^-) \gamma\), is called accidentally. This functionality is going to be deprecated and cosHelicityAngleMomentumPi0Dalitz is recommended to use instead.

  • The variable cosHelicityAngleMomentumPi0Dalitz is extended to the case of pi0 two-body decay, where a photon is reconstructed from the photon conversion, \(\pi^0 \to \gamma (\to e^+ e^-) \gamma\).

1.2.6. Changes in the analysis package in light-2309-munchkin#

Bugfix of ParticleListManipulator for particleLists loaded from KLMClusters#

  • Fixed a major bug in the ParticleListManipulator used in functions such as copyList, fillParticleList and other modularAnalysis functions. The bug occurred when creating a particle list from two or more types of mdst objects, for example, K_L0 loaded from ECL clusters and KLM clusters. When a new particle list was created from original lists, mdst objects with the same mdst index as other mdst objects were skipped and not added to the new particle list, even if the mdst types were different. This mainly impacted the K_L0 or neutron particle lists, resulting in a significant drop in particles loaded from KLM clusters. No such impact has been observed for other types of particle lists.

FEI#

  • Applied the mass constraint to pi0 candidates in FEI. Please note that the FEI calibration on MC15ri is performed without applying pi0 mass constraint and without loosening the gamma energy threshold, which has been implemented since light-2305-korat. So, either use FEI skims (recommended), or run FEI with an older release until new FEI calibration with MC16 will be available.

Photon Energy Resolution#

  • Added ECLPhotonEnergyResolution to analysis database objects. This module reads the relative energy resolution calculation results from the Photon Energy Resolution Calibration workflow and creates a corresponding database object. The new method GetCovarianceMatrix3x3FromCluster in ClusterUtils replaces the former calculated values if the cluster properties (\(E\), \(\phi\), \(\theta\)) are within valid binning and leaves them unchanged otherwise. ClusterUtils triggers the replacement if a globaltag with a valid ECLPhotonEnergyResolution payload is present and without further user input.

Particle weights#

  • Added a new utility class PostProcessingParticleWeighting for post-processing particle weighting. This class reads the weights from the database for the specified experiment and run numbers and the weight-table variables.

Variables#

  • Added the variables PxHER, PyHER, PzHER and PxLER, PyLER, PzLER which are the nominal beam momenta derived from the CollisionInvariantMass, CollisionBoostVector and CollisionAxisCMS payloads

  • Added the variables mcPxHER, mcPyHER, mcPzHER and mcPxLER, mcPyLER, mcPzLER which are the truth momenta of colliding electron/positron for the event. These are available only for MC. For EvtGen and BabaYaga generators these variables have dummy value for MC15 or older.

1.2.7. Changes in the skim package in light-2401-ocicat#

New skims#

  • SL&ME:

  • TDCPV:

  • Tau:

    • 18570800: \(e^+ e^- \to \tau^+ \tau^-, \tau \to K_s \pi X\)

  • Bhadronic:

    • 19140500: \(B^{+} \to \eta^{'} K^{*+}\)

    • 14160200: \(B^{0} \to D_{s1}^{+} D^-\)

    • 14160201: \(B^{0} \to D_{s0}^{*+} D^{-} (\to K^+ \pi^- \pi^-)\)

    • 14121400: \(B^{0} \to \overline{D}^{*-} (\to \bar{D}^{0} (\to K^+ \pi^-) \pi^-) \pi^+ \pi^0\)

    • 14161400: \(B^{+} \to D^{*-} (\to \overline{D}^{0} (\to K^+ \pi^-) \pi^-) \pi^+ \pi^+ \pi^0\)

    • 14121500: \(B^{0} \to \overline{D}^{*-} (\to \bar{D}^{0} (\to K^+ \pi^-) \pi^-) \eta (\to \gamma \gamma) \pi^+\)

    • 14121600: \(B^{+} \to \bar{D}^{0} (\to K^+ \pi^-) \eta (\to \gamma \gamma) \pi^+\)

    • 14121601: \(B^{+} \to \bar{D}^{0} (\to K^+ \pi^-) \pi^+ \pi^0\)

  • Dark & low multi:

    • 18530600: \(e^{+} e^{-} \to \gamma \pi^{+} \pi^{-}\), \(e^{+} e^{-} \to \gamma \mu^{+} \mu^{-}\)

    • 18020600: \(e^+ e^- \to A' (\to \mu^+ \mu^-) h'\)

    • 18020700: \(e^+ e^- \to A' (\to e^+ e^-) h'\)

  • Charm:

    • 17230700: \(D^{*} \to D^{0} \pi\)

    • 17260100: \(D^{+} \to K^{+} \omega (\to \pi^+ \pi^- \pi^0)\), \(D^{+} \to \pi^{+} \omega (\to \pi^+ \pi^- \pi^0)\) (and CC)

    • 17260200: \(D_{s}^{+} \to K^{+} \omega (\to \pi^+ \pi^- \pi^0)\), \(D_{s}^{+} \to \pi^{+} \omega (\to \pi^+ \pi^- \pi^0)\) (and CC)

    • 17220100: \(D^+ \to \pi^+ e^- e^+\)

    • 17220200: \(D^+ \to \pi^+ \mu^- \mu^+\)

    • 17220300: \(D^+ \to \pi^+ K^- K^+\)

    • 17230900: \(\Lambda_c^+ \to \Sigma^+ h^+ j^-\)

    • 17231000: \(\Xi_c^+ \to p \pi^+ K^-\)

    • 17231100: \(\Xi_c^+ \to \Sigma^+ h^+ j^-\), \(\Xi_c^+ \to \Xi^- h^+ j^-\)

    • 17231200: \(\Xi_c^0 \to \Lambda^0 h^+ j^-\)

    • 17232000: \(D^{+} \to \pi^+ \pi^0\)

  • Quarkonium:

    • 15460400: \(\Upsilon(1S,2S,3S) \to l^+ l^- (l = e, \mu)\)

    • 16460200: \(J/\psi \to l^+ l^- (l = e, \mu)\), \(\psi(2S) \to l^+ l^- (l = e, \mu)\)

Modified skims#

1.2.8. Changes in the mva package since release-08-00#

  • Added an interface for pyTorch. Please see mva/examples/torch/simple.py for a basic example for binary classification. As with all other supported python interfaces the interface is designed for users to override the get_model(...), begin_fit(...), partial_fit(...), and apply(...) methods in their training script to define their models and training loop as needed.

  • With externals v02-02-01 we moved to tensorflow v2.16 which contains keras v3. The keras interface and examples have been updated to use this new version. The new interface is not backwards compatible for weightfiles created with the previous interface.

1.3. Changes since release-07#

New and updated packages in externals

Table 1.2 New and updated packages in externals from v01-12-01 to v02-00-02#

package

old version

new version

clhep

2.4.1.3

2.4.6.2

Geant4

10.06.p03

11.1.1

Vc

1.4.1

1.4.3

VecCore

0.6.0

0.8.0

VecGeom

1.1.5

1.2.1

VGM

4.6

5.0

WHIZARD

3.1.2

beautifulsoup4

4.11.1

pydata-sphinx-theme

0.8.1

soupsieve

2.3.2.post1

sphinx-codeautolink

0.15.0

Authentication for Conditions Database using JWT

The authentication method that allows to upload payloads to the Conditions Database and to modify existing global tags has been updated. It now uses JSON web tokens that are created and verified based on the B2MMS username and the associated password. More information can be found in the section b2conditionsdb: Conditions DB interface.

1.3.1. Changes in the analysis package in light-2307-laperm#

Modules#

  • Added support of decay string functionality to MVAExpert module, so it can be used for daughter particles. The first MVA to support this feature is getNbarIDMVA.

Variables#

1.3.2. Changes in the analysis package in light-2305-korat#

Modules#

  • Added an option selectedDaughters to specify daughters to apply weightings in ParticleWeighting.

Geometric matching of KL and neutrons to ECL clusters#

Variables#

Reconstruction of V0s#

  • stdKshorts and stdLambdas: removed a pre-vertex-fit cut on the invariant mass of K_S0/Lambda0 candidates found by the V0Finder module. The cut was not useful, as V0Finder already applies a cut on its candidates, and could discard signal candidates with very displaced vertex due to the K_S0/Lambda0 vertex being assumed at the origin before the fit, affecting the invariant mass computation. The post-fit cut on the invariant mass is still present. Cuts on K_S0/Lambda0 candidates found by reconstructDecay are still the same as before.

Command line tools#

  • The command line tool b2help-variables is extended to get a description of specific variables and to display only their data.

FEI#

  • The gamma cuts for Belle II inside FEI are loosened (to match goodBelleGamma i.e., 0.09 –> 0.05 MeV in barrel and 0.16 –> 0.15 MeV in backward endcap) starting this release. Hence, the FEI calibration performed with MC15ri are not valid anymore if FEI is reconstructed with this release or later. So, either use FEI skims (recommended), or run FEI with an older releases until new FEI calibration with MC16 will be available.

1.3.3. Changes in the analysis package in light-2303-iriomote#

Modules#

  • Fixed a bug in the EventShapeCalculator module and modularAnalysis.buildEventShape that was introduced in light-2302-genetta. Meaningless values were provided for the event-shape variables.

  • Fixed a bug on the findMCDecay that creates duplicated particles which differ only in the order of their daughters.

  • Fixed a bug of the TreeFit that affected the initialization of vertex position that is calculated from two track helices. This bug mainly affected particles that decay far from the IP, specifically V0 particles such as K_S0. The bugfix recovers the inefficiency of those particles.

Variables#

  • Added new charged PID variables kaonIDNN and pionIDNN representing the probability for the given hypothesis, calculated from a neural network that uses high-level information such as PID likelihoods, track momentum and charge. The probabilities are unaffected by momentum updates in vertex fits or the application of the track momentum scale correction.

1.3.4. Changes in the analysis package in light-2302-genetta#

Charm Flavor Tagger#

  • The Charm Flavor Tagger algorithm has been added in analysis/scripts/charmFlavorTagger.py. This allows for the determination of the signal flavor of neutral D mesons with information from the ROE. The CFT output, qr, is saved in the extraInfo field of the signal particle.

MCMatching of K_S0#

  • Modified the MCMatching algorithm for the K_S0 particle. The MCError flags of missing daughters (e.g. c_MissMassiveParticle. please see Error flags) was not assigned to the K_S0. It was not a problem for the \(K_S^0 \to \pi^+ \pi^-\) process, because K_S0 fails to be matched to the MCParticle when one of pi+ is missed. However, when the K_S0 decays interacting with the detector material, it can have many daughters. If the K_S0 is reconstructed from two pions of these daughters, it was considered as “signal” (i.e. isSignal==1) even though its kinematics was wrongly reconstructed. We modified the MCMatching algorithm and assigned the MCError flags of missing daughters on the K_S0 properly. The most of “fake-signal” K_S0 candidates are now correctly identified as background (i.e. isSignal==0).

  • There remains a small fraction of “fake-signal” candidates that decay interacting with the detector material but having only two pions. The effect is expected to be small, for example, that is about 0.0005% in the tau-LFV process, \(\tau^- \to \ell^- K_S^0 K_S^0\). You can identify such events with the mcSecPhysProc variable that returns the secondary process of the Geant4, namely countFSPDaughters(mcPrimary==0 and mcSecPhysProc!=201)>0.

Graph Neural Network based Flavor Tagger#

  • The Graph Neural Network based Flavor Tagger is available from the basf2. The new options useGNN and identifierGNN are added to flavorTagger.flavorTagger. If useGNN is set to True and identifierGNN is set to the trained weight file name, the qr value is stored in the alias variable qrGNN. One can find the information on the identifier and the global tag which includes the payload on the TDCPV WG Confluence Page.

Modules#

  • New function modularAnalysis.estimateAndAttachTrackFitResult and module TrackFitResultEstimator are added. It creates a TrackFitResult from the momentum of the Particle and creates a relation between them. The main use case is the tracking efficiency study by extrapolating the recoil particle to the detector surfaces.

  • Removed the prepending of the analysis global tag inside the modularAnalysis functions, lowEnergyPi0Identification, getBeamBackgroundProbability, getFakePhotonProbability, and getNbarIDMVA, so that the priority of global tags gets clearer.

  • Fixed a bug on the findMCDecay that the decay of K_S0 cannot be specified.

  • Added an option usePhotonOnlyOnce to the function modularAnalysis.correctBremsBelle(). The default of the option is False so the backward compatibility. If the option is set to True, a photon is used for only the closest charged particle in the input ParticleLists.

  • Modified parameters for the functions modularAnalysis.fillParticleList() and modularAnalysis.fillParticleLists() so that they no longer contain parameters that load the beam background or fake photon (formerly hadronic split-off) MVA.

  • Parameters for stdPhotons, stdPi0s and stdHyperons have been changed in both name and type. The new parameters are called beamBackgroundMVAWeight and fakePhotonMVAWeight, and they are now strings to specify which weight files should be used. For run-independent training use MCri while for run-dependent training use MCrd. More information on the new weights can be found on the Neutrals Performance Confluence Page.

  • Added an optional parameter biasCorrectionTable to stdHyperons so that users can specify a table name with photon energy bias corrections for the stdXi0 particle list. The latest information on the bias correction can be found on the Neutrals Performance Confluence Page.

  • Added a parameter updateAllDaughters to stdKshorts and stdLambdas so that users can instruct the vertex fitter to update the variables of the V0 daughters. The parameter is False by default for backward compatibility.

  • New function modularAnalysis.fillParticleListFromChargedCluster is added to load the neutral final-state-particles from the ECL and KLM clusters that are being matched with the Track of input ParticleList.

  • Added the argument ignoreFromVertexFit in vertex.treeFit() to ignore particles to determine the vertex position. The kinematics of the particles are treated as constant. The ignoreFromVertexFit option is expected to be used to perform fit without K_L0 or mimic a K_S0 or a pi0 with a control channel by ignoring a track for vertex fit. While, treatAsInvisible will ignore both for vertex and kinematics. The treatAsInvisible option is expected to recover an invisible particle’s kinematics from beam-constraint.

  • Fixed a bug of the TreeFit with the Bremsstrahlung correction tools, modularAnalysis.correctBrems and modularAnalysis.correctBremsBelle. The position uncertainties of the brems-corrected particle were wrongly assigned. This bug does not affect the kinematics very much but induces a bias on the position and its uncertainty.

  • Allowed -o argument of basf2 to override the output ROOT filename of VariablesToNtuple, VariablesToEventBasedTree, and VariablesToHistogram modules, e.g. basf2 script.py -o new.root. If the modules are called more than once, the override will be applied to all output files. If you need to save multiple TTrees with the same name, please use the new argument filenameSuffix of modularAnalysis.variablesToNtuple to differentiate the output files. The filenameSuffix is appended to the output ROOT file name even if the -o argument is used.

  • Fixed a bug on NeutralHadron4MomentumCalculator that misplaces the neutral hadron daughter’s index after calculating its momentum.

  • New function modularAnalysis.extractParticlesFromROE and module ParticleExtractorFromROE are implemented. The function extracts Particle objects that belong to a ROE and fills them in ParticleLists. It is available both in the for_each loop over the ROEs (e.g. roe_path) and the main path if the signal side ParticleList has only one candidate.

  • Added a module BelleNbarMVA that applies nbarMVA for Belle, a DNN trained with ECLCluster variables for discrimination of anti-neutrons against photons. Anti-neutron-like particles have scores closer to 1; photon-like particles have scores closer to 0.

  • New interface for module TrackIsoCalculator that allow creating a single module instance for multiple detectors. The module now also calculates an isolation score per particle according to a new definition (see Variables).

  • Fixed a bug of the TreeFit with the particle loaded from the KLMCluster. The mass of the particle, which should have been fixed with the PDG value, was floated in the TreeFit. Now it is fixed and the fit results get more reasonable. However, since the covariance matrix of the KLMCluster is not properly set yet, the performance of the TreeFit with these particles is not guaranteed.

  • Added an option overwriteRank to the modularAnalysis.rankByHighest and modularAnalysis.rankByLowest to overwrite the extraInfo of rank when the particle has already the extraInfo.

Variables#

  • New variable momentaTripleProduct is added for the searches for CP violation via T-odd correlations in multi-body decays.

  • Added the variables EcmsMC and totalEnergyMC, which return the true value of \(\sqrt{s}\) and the true sum of the energies of all generated particles, respectively.

  • Deprecated hadronicSplitOffSuppression which has been replaced with fakePhotonSuppression

  • New variables countFSPDaughters and countDescendants are added to get number of daughters which passes the given selection.

  • New meta-variables originalParticle and originalDaughter are added to get the variable of the original particle from which the given particle or the specified daughter. One can get the variables of the daughters particles before they are updated by the vertex fit, for example.

  • New variables helixExtThetaOnDet and helixExtPhiOnDet are added. They returns theta and phi of extrapolated helix parameters on the detector surface given by the argument.

  • New variable clusterBestMatchedMCParticle is added to access the MCParticle that is related to the ECLCluster but not to the Particle object.

  • New variable minET2ETIsoScoreAsWeightedAvg that returns a particle’s isolation score based on a weighted average of inverse distances at each input detector layers.

1.3.5. Changes in the analysis package in light-2212-foldex#

ParticleList dataobject#

  • Added new member variable isReserved to prevent the reserved ParticleLists (:all labeled final-state particles, and :V0 labeled V0-particles) from being changed. While some modules have already had protection not to edit the reserved lists, e.g. ParticleSelector and ParticleListManipulator, the other modules editing the ParticleList had the potential to introduce critical bugs. If one’s script breaks due to the update since the light-2212-foldex, please modify the script not to edit the reserved ParticleLists by copying them with other names beforehand.

Signal embedding#

  • New modules UdstListFilter and PostMergeUpdater for signal embedding which remove extra mdst objects and merge events together, respectively.

  • New scripts udst_purge_list and embedding_merge to enable signal embedding.

  • A python script analysis/examples/embedding/embedding.py which illustrates the usage of the scripts.

Variables#

stdCharged#

  • Added working points for uniform pion-lepton fake rate.

stdPhoton and stdPi0#

  • Added an optional parameter biasCorrectionTable to stdPhotons and stdPi0s that allows the user to specify a table name for the photon energy bias correction. Providing a table name will automatically apply the bias correction to all particles in the particle lists. This is available for all list types. The latest information on the bias correction can be found on the Neutrals Performance Confluence Page.

Modules#

  • Added an option signalSideParticleList to the function modularAnalysis.variablesToNtuple() to store the index of the signal-side particle. This option is only valid when one calls the function in a for_each loop over the RestOfEvent. The branches __signalSideCandidate__ and __nSignalSideCandidates__ are added to the TTree by setting the option. An example of usage can be found in analysis/examples/VariableManager/variablesToNtuple.py.

  • Fixed a bug on the BestCandidateSelection that keeps only less than the expected number of candidates when allowMultiRank=True and numBest>0. Now the module works as documented.

  • Added an option allowAnyParticleSource to the function modularAnalysis.reconstructDecayWithNeutralHadron() to let users apply the mother-mass constraint on a charged particle.

  • The findMCDecay function is fixed so that all output particles have isSignal==1. New option appendAllDaughters is added so that one can append not only the daughters described in the decay string but all daughters of the matched MCParticle. Another new option skipNonPrimaryDaughters is added to skip secondary particles when appendAllDaughters=True.

  • The ParticleLists labeled :MC, which were created by default when the fillParticleListFromMC is called, are no longer created automatically by the function. Also, the protection that was previously applied to the list labeled “:MC” has been removed.

1.3.6. Changes in the analysis package in light-2210-devonrex#

Modules#

  • Fixed a bug in KFit when the daughters’ kinematics are updated (daughtersUpdate=True). The mother’s momentum was calculated at the updated vertex position, while the updated daughters’ momenta were calculated at the original position. This caused an inconsistency in the momentum between the mother and the sum of the daughters. Now the daughters’ momenta are also calculated at the updated vertex position and the inconsistency is resolved.

  • Changed TreeFit and KFit to be compatible with the momentum scaling factor. The daughters’ kinematics as member variables of the Particle class are updated by being divided by the momentum scaling factor. However, the vertex fitters do not change the momentum scaling factor. So, the kinematics variables now return correct values.

1.3.7. Changes in the analysis package in light-2209-calico#

MVA-based pi0 identification#

Variables#

  • Extended the variable useAlternativeDaughterHypothesis to variables calculated from daughters’ 4-momentum, e.g. cosHelicityAngleMomentum.

  • Added metavariable to retrieve the distance to the closest neighbour at a detector surface (based on track helix extrapolation): minET2ETDist.

  • Added metavariable to retrieve any variable associated to the closest neighbour at a detector surface from the reference particle list: minET2ETDistVar.

  • Added metavariable to retrieve the particle isolation score (based on minET2ETDist and, optionally, on PID detector weights): minET2ETIsoScore.

Modules#

Database Objects#

  • New database object class PIDDetectorWeights to store PID detector weights per standard charged particle, binned in p and theta.

Modular Analysis Function#

  • Added a check to the applyEventCuts function to make sure that only event-based variables are used in the selection. The decomposition of meta variables can be tricky, so a new argument metavariables is added to extend the list of meta variables which should be decomposed and should themselves be ignored in the check.

  • Added the argument treatAsInvisible in vertex.treeFit() to ignore particles in the fit.

  • Updated the calculateTrackIsolation helper. Now, it returns a list of metavariables (minET2ETDist, minET2ETIsoScore, optionally minET2ETDistVar as well) correctly configured according to the helper input options. Users can then create aliases in their own steering script: see analysis/examples/VariableManager/trackIsolationVariables.py for an example.

  • The default argument of flavorTagger.flavorTagger() is updated so that the weight files trained with MC15ri will be used as default. If one wants to use the previous weight files, the arguments should be set as follows: maskName='all' and prefix=''.

  • With the vertex.kFit(), the daughters’ kinematics are updated after the vertex fit, even when fit daughters are selected using decay_string and daughtersUpdate=True.

  • The new argument ignoreMotherFlavor has been added to the function mergeListsWithBestDuplicate. It allows to ignore the flavor of the mother particle when identifying duplicates.

Particle dataobject#

  • Added a replaceDaughter method to Particle. It allows you to replace the daughter of a Particle with another Particle while keeping the original order of the daughters. This is important to ensure the decay string will work correctly after replacing a daughter.

  • Added a new Particle source NoMDSTSource to Particle. It can be used for Particles that do not come from any mdst source.

1.3.8. Changes in the analysis package in light-2207-bengal#

Variables#

Modules#

  • Fixed a bug in the modularAnalysis.fillParticleListFromROE(). The relationship of the pdg code between the loaded particle and the source particle was inconsistent when the loaded particle was not a self-conjugated particle.

1.3.9. Changes in the b2bii package since release-07-00#

Conversion#

  • Added an option to copy anti-n0:mdst` from ``gamma:mdst

1.3.10. Changes in the mva package since release-07-00#

  • Added support of decay string functionality to MVAExpert module, so it can be used for daughter particles.

  • Updated the argument overwriteExistingExtraInfo of MVAExpert and MVAMultipleExperts modules so that one can more flexibly choose to keep or overwrite existing information. Instead of a boolean users should now pass one of (-1/0/1/2). An existing extra info with the same name will be overwritten if the new value is lower / will never be overwritten / will be overwritten if the new value is higher / will always be overwritten (option = -1/0/1/2).

1.4. Changes since release-06#

New and updated packages in externals

Table 1.3 New and updated packages in externals from v01-10-02 to v01-12-01#

package

old version

new version

astyle

2.05.1

3.1

boost

1.72

1.78

cmake

3.19.7

3.22.2

curl

7.75.0

7.77.0

eigen

3.3.9

3.4.0

EOS

0.3.3

removed

Frugally-deep

0.15.19-p0

FunctionalPlus

0.2.18-p0

gcc

10.2

11.2

GDB

10.1

10.2

Hammer

1.1.0

1.3.0

Libxml2

2.9.10

2.9.13

Libxslt

1.1.34

1.1.35

Madgraph

2.7.2

3.4.0

Millepede-II

V04-09-00

V04-10-00

nlohmann/json

3.9.1

3.10.5

ROOT

6.24.0

6.24/06

RooUnfold

3.0.0

XRootD

5.1.1

5.4.3

zlib

1.2.11

1.2.12

Integer Set Library

0.21

0.24

cppcheck

2.3

2.6

absl-py

0.12.0

1.3.0

aiohttp

3.8.1

aiosignal

1.2.0

amply

0.1.4

removed

anyio

3.3.0

removed

argon2-cffi

20.1.0

21.3.0

argon2-cffi-bindings

21.2.0

async_generator

1.10

removed

async-timeout

4.0.2

attrs

21.2.0

21.4.0

autopep8

1.5.7

1.6.0

awkward

1.2.2

1.7.0

bleach

3.3.0

4.1.0

boost-histogram

1.0.2

1.2.1

cachetools

4.2.2

4.2.4

certifi

2020.12.5

2021.10.8

cffi

1.14.5

1.15.0

chardet

4.0.0

removed

charset-normalizer

2.0.10

cheroot

8.5.2

8.6.0

CherryPy

18.6.0

18.6.1

cloudpickle

1.6.0

2.0.0

ConfigArgParse

1.4

1.5.3

connection_pool

0.0.3

cryptography

3.4.7

36.0.1

cycler

0.10.0

0.11.0

Cython

0.29.23

0.29.26

debugpy

1.5.1

decorator

5.0.7

5.1.1

dgl

0.6.1

eoshep

1.0.1

filelock

3.0.12

3.4.2

flake8

3.9.2

4.0.1

flatbuffers

1.12

22.9.24

fonttools

4.28.5

frozenlist

1.3.0

fsspec

2022.5.0

gitdb

4.0.7

4.0.9

GitPython

3.1.14

3.1.26

google-auth

1.30.0

2.3.3

google-auth-oauthlib

0.4.4

0.4.6

idna

2.10

3.3

imagesize

1.2.0

1.3.0

importlib-metadata

4.10.0

importlib-resources

5.4.0

ipykernel

5.5.4

6.6.1

ipython

7.23.1

7.31.0

ipywidgets

7.6.3

7.6.5

jaraco.collections

3.3.0

3.5.1

jaraco.functools

3.3.0

3.5.0

jaraco.text

3.5.0

3.6.0

jedi

0.18.0

0.18.1

jeepney

0.7.1

Jinja2

2.11.3

3.0.3

jira

2.0.0

3.1.1

joblib

1.0.1

1.1.0

json5

0.9.6

removed

jsonschema

3.2.0

4.3.3

jupyter-client

6.1.12

7.1.0

jupyter-core

4.7.1

4.9.1

jupyter-server

1.10.2

removed

jupyterlab

3.1.9

removed

jupyterlab-server

2.7.2

removed

jupyterlab-widgets

1.0.0

1.0.2

jupytext

1.11.2

1.13.6

keras

2.4.3

2.10.0

keyring

23.5.0

kiwisolver

1.3.1

1.3.2

libclang

14.0.6

lightgbm

3.3.2

lxml

4.6.3

4.7.1

Markdown

3.3.4

3.3.6

MarkupSafe

1.1.1

2.0.1

matplotlib

3.4.2

3.5.1

matplotlib-inline

0.1.2

0.1.3

mdit-py-plugins

0.2.8

0.3.0

metakernel

0.27.5

0.28.2

more-itertools

8.7.0

8.12.0

multidict

6.0.2

mysql-connector-python

8.0.24

8.0.27

nbclassic

0.3.1

removed

nbclient

0.5.3

0.5.9

nbconvert

6.0.7

6.4.0

nbsphinx

0.8.4

0.8.8

nest-asyncio

1.5.1

1.5.4

networkx

2.7.1

notebook

6.4.3

6.4.6

numexpr

2.7.3

2.8.1

numpy

1.19.5

1.20.1

oauthlib

3.1.0

3.1.1

packaging

20.9

21.3

pandas

1.2.4

1.3.5

pandocfilters

1.4.3

1.5.0

parso

0.8.2

0.8.3

pbr

5.6.0

removed

Pillow

8.2.0

9.0.0

pip

21.1.1

21.3.1

pluggy

0.13.1

1.0.0

portend

2.7.1

3.1.0

prometheus-client

0.10.1

0.12.0

prompt-toolkit

3.0.18

3.0.24

protobuf

3.16.0

3.19.3

psutil

5.8.0

5.9.0

PuLP

2.4

2.6.0

py

1.10.0

1.11.0

pycodestyle

2.7.0

2.8.0

pycparser

2.20

2.21

pyDeprecate

0.3.1

pydocstyle

6.0.0

6.1.1

pyflakes

2.3.1

2.4.0

Pygments

2.9.0

2.11.2

PyJWT

2.1.0

removed

pylama

7.7.1

8.3.7

pyparsing

2.4.7

3.0.6

pypmc

1.2

pyrsistent

0.17.3

0.18.0

pytest

6.2.4

6.2.5

python-dateutil

2.8.1

2.8.2

python-gitlab

3.9.0

pytorch-lightning

1.5.6

pytz

2021.1

2021.3

PyYAML

5.4.1

6.0

pyzmq

22.0.3

22.3.0

qtconsole

5.1.0

5.2.2

QtPy

1.9.0

2.0.0

requests

2.25.1

2.27.1

requests-unixsocket

0.2.0

removed

rsa

4.7.2

4.8

scikit-learn

0.24.2

1.0.2

scipy

1.6.3

1.7.3

SCons

4.1.0post1

4.3.0

seaborn

0.11.1

0.11.2

SecretStorage

3.3.1

Send2Trash

1.5.0

1.8.0

setuptools

56.2.0

60.5.0

smart-open

5.0.0

5.2.1

smmap

4.0.0

5.0.0

snakemake

6.3.0

6.13.1

sniffio

1.2.0

removed

snowballstemmer

2.1.0

2.2.0

Sphinx

4.0.0

4.3.2

sphinx-book-theme

0.3.3

sphinxcontrib-htmlhelp

1.0.3

2.0.0

sphinxcontrib-programoutput

0.17

sphinxcontrib-serializinghtml

1.1.4

1.1.5

sweights

1.0.0

tables

3.6.1

3.7.0

tempora

4.0.2

5.0.0

tensorboard

2.5.0

2.10.1

tensorboard-plugin-wit

1.8.0

1.8.1

tensorflow

2.4.1

2.10.0

tensorflow-estimator

2.4.0

2.10.0

tensorflow-io-gcs-filesystem

0.27.0

tensorflow-probability

0.12.2

0.15.0

terminado

0.9.4

0.12.1

testpath

0.4.4

0.5.0

threadpoolctl

2.1.0

3.0.0

toposort

1.6

1.7

torch

1.8.1

1.11.0

torch-geometric

2.0.4

torch-scatter

2.0.9

torch-sparse

0.6.14

torchaudio

0.8.1

0.11.0

torchmetrics

0.9.3

torchvision

0.9.1

0.12.0

tqdm

4.60.0

4.62.3

traitlets

5.0.5

5.1.1

uncertainties

3.1.5

3.1.6

uproot

4.0.7

4.1.9

urllib3

1.26.4

1.26.8

websocket-client

1.2.1

removed

Werkzeug

1.0.1

2.0.2

wheel

0.36.2

0.37.1

widgetsnbextension

3.5.1

3.5.2

xgboost

1.6.1

yarl

1.7.2

zipp

3.7.0

zstandard

0.15.2

0.16.0

Simplified arguments to modularAnalysis.inputMdst() and modularAnalysis.inputMdstList().

The arguments of modularAnalysis.inputMdst() and modularAnalysis.inputMdstList() have been changed a little. You no longer need to specify “default”, it’s done automatically.

The following code lines need to be changed from:

# old
import modularAnalysis as ma
ma.inputMdst("default", "/path/to/your/file.root", path=mypath)

# or
ma.inputMdst("Belle", "/path/to/your/file.root", path=mypath)

To:

# new
import modularAnalysis as ma
ma.inputMdst("/path/to/your/file.root", path=mypath)

# or
ma.inputMdst("/path/to/your/file.root", path=mypath, environmentType="Belle")

And similarly for modularAnalysis.inputMdstList().

Warning

We no longer support MC5-10 files.

Breaking of backward compatibility for kinematic variables

The floating-point members of the Particle class (invariant mass, momentum and position components) are now stored as doubles and no longer as floats. Previously, in some edge cases like for ISR photons, in particular the energy calculation could suffer from a cancellation of significant digits. This change might slightly modify the values of (derived) kinematic variables, however it should be below most analysts sensitivity.

Track time

When reconstructing data with release-07 the new variable trackTime becomes available, which is computed by the TrackTimeEstimatorModule. This track time is the average time of the SVD clusters attached to the track minus the SVDEventT0.

1.4.1. Changes in the analysis package in light-2207-bengal#

PID Calibration Weight Tools#

  • Added the module scripts/pidDataUtils.py to provide a set of tools for preparing and analyzing data with weights

  • Added the script scripts/pidTrainWeights.py to train a set of calibration weights on a dataset

  • Added an example for training a set of weights and using them in an analysis in analysis/examples/PIDCalibration

Variables#

  • Added the variable pionID_noARICHwoECL which returns pionID without using ARICH likelihood for tracks without associated ECL cluster.

  • Added the variable kaonID_noARICHwoECL which returns kaonID without using ARICH likelihood for tracks without associated ECL cluster.

  • Added the metavariable binaryPID_noARICHwoECL which returns binary PID without using ARICH likelihood for tracks without associated ECL cluster.

1.4.2. Changes in the analysis package in light-2205-abys#

Variables#

  • Added the variable nPrimaryMCParticles which returns the number of primary MC particles.

  • Added the variable nVirtualPrimaryMCParticles which returns the number of virtual primary MC particles.

  • Added the variable nInitialPrimaryMCParticles which returns the number of initial primary MC particles.

  • Changed the nCompositeDaughters so that one can specify the PDG code of composite daughters. If no argument is given, it works same as before. Please note that brackets have to appended even if no argument is given i.e. nCompositeDaughters().

  • Added nCompositeAllGenerationDaughters which works similar with nCompositeDaughters but it includes all generations of daughters.

  • Added optional argument to helixExtTheta, helixExtPhi to allow performing the extrapolation using the mass hypothesis that provides the best track fit result, regardless of the particle object’s PDG.

  • Users can clear aliases by using variables.clearAliases(), which is useful when processing several independent Path objects in a single basf2 session, e.g. b2luigi job submission.

  • Added the variable cosAngleBetweenMomentumAndBoostVector which returns the cosine of angle between momentum and boost vector.

Modules#

  • Added an option basketsize to modularAnalysis.variablesToNtuple() to set the size of TBaskets in the output ntuple in bytes.

  • Added an argument reference_list_name to modularAnalysis.calculateTrackIsolation(). The reference ParticleList is used as reference to calculate the distance for the input ParticleList given by input_list. By default, the :all list of the same particle type as the input_list is used.

  • The TrackIsoCalculator and ChargedPidMVA modules now support decay string grammar, which allows to run these modules on daughter or granddaughter particles.

  • In the CDC volume, the calculation of track-to-closest-neighbour distance in the TrackIsoCalculator module now is done for 9 surface layers (was only at the entry surface). Also, the definition of distance has been changed to adapt to each sub-detector’s geometrical segmentation.

  • In the TrackIsoCalculator module, added an option to calculate the distance using the track fit result of the most probable mass hypothesis for the particles in the reference list.

MC Truth Information#

MVA-based K_S0 Selector#

  • Added a new function ksSelector.ksSelector(), which applies MVA-based selection of K_S0 particleList

  • Added a example for usage and training of ksSelector at B2A910, B2A911, and B2A912

1.4.3. Changes in the analysis package in light-2203-zeus#

Lepton ID Analysis Helper#

  • Added stdCharged.stdLep() helper to build standard lepton lists for a number of supported working points and LID methods (Likelihood ratio or BDT, global or binary). The helper also decorates the particles with the recommended Data/MC corrections and associated uncertainties for efficiency and fake rates, based on the chosen GT.

Vertex Fitting#

  • Fixed the calculation of kinematics after vertex KFit with selected particles. Disabled the feature of selecting fit particles for KFit except for vertex fit.

Variables#

  • Removed EventType, which was deprecated for more than a year since light-2012-minos.

  • Deprecated the variable daughterInvariantMass, which duplicates daughterInvM.

  • Added new Flavor Tagger related meta-variables to use ROE-mask correctly, pMissTag, cosTPTO, ptTracksRoe, pt2TracksRoe, isMajorityInRestOfEventFromB0, isMajorityInRestOfEventFromB0bar, hasRestOfEventTracks, and isRelatedRestOfEventMajorityB0Flavor. Added an argument maskName to BtagToWBosonVariables.

  • Added the variable trackFitCovariance, which returns the track fit covariance matrix element corresponding to the two provided indices.

Modules#

  • The TrackingMomentumScaleFactors module can now also smear track momenta based on phase-space dependent smearing factors which are read from payloads according to parameters payloadName and smearingFactorName. The corresponding wrapper function smearTrackMomenta has been added. The argument tableName has been renamed as payloadName in scaleTrackMomenta.

  • Added an option skipNonPrimary to fillParticleListFromMC and fillParticleListsFromMC.

  • Added an argument prefix to flavorTagger.flavorTagger() so that one can choose the version of weight files.

1.4.4. Changes in the analysis package in light-2112-uranus#

Vertex Fitting#

  • Added a new fit-type option recoilmass to function vertex.kFit(). The kinematic fit is performed with a constraint of a recoil mass specified by the option recoilMass.

  • Fixed a bug in fourC and massfourC kfit. The mass of final-state particles were not fixed in old version.

Variables#

  • Added the variable TracksLBoostChi2, which returns the change of the track positions in the boost direction after a vertex KFit in terms of a \(\chi^2\) value.

Modules#

  • A cut provided to the ParticleCombiner was always applied to the daughter particle combination, even when choosing one of the two recoil options. Now it is applied to the created mother particle as written in the documentation. This changes the behavior of reconstructRecoil and reconstructRecoilDaughter.

  • The ParticleKinematicFitter module has been extended to support the fit in which an additional unmeasured photon is assumed along HER/LER. To use the feature, one has to set addUnmeasuredPhoton and fixUnmeasuredToHER (fixUnmeasuredToLER) to true.

  • Added a Pi0VetoEfficiencySystematics module to obtain pi0 veto efficiency Data/MC weights, their statistical and systematic errors as extrainfo variables for a given hard photon and the veto threshold. addPi0VetoEfficiencySystematics is a corresponding wrapper function. The allowed range of the veto threshold is extended to (0.10,0.99).

PID Priors#

  • Added analysis-train-priors tool that trains a pytorch model for calculating PID priors using an ntuple containing momentum and cos(\(\theta\)) information. It generates a second order combination of the previously mentioned variables along with transverse momentum giving a total of nine features and trains the model using a taylor series of cross entropy loss to reduce the effect of outliers on the training.

  • For evaluating priors, evalPriors module containing the Priors class has been added to calculate these using the trained model. Additionally, posteriors can also be obtained by providing PID’s from the detector signals.

1.4.5. Changes in the analysis package in light-2110-tartarus#

Variables#

  • The no ROE mask case in the ROE-related variables has now a different name: "all" instead of an empty string "". Similarly to the ParticleLoader’s all particle lists, this name is reserved, user cannot create a mask with this name. If maskName argument is not specified in the ROE-related metavariables, it will default to all automatically, but in case of metavariables with many arguments user will be required to call it explicitly, e.g. nROE_Charged(all, 321).

  • Added v0DaughterNRemovedHits which returns the number of track hits removed in V0Finder.

  • Added hadronicSplitOffSuppression which returns the output of an MVA that uses shower-related variables to distinguish between true photons and hadronic split-offs.

  • The variable beamBackgroundProbabilityMVA now has a new name: beamBackgroundSuppression. The weight files for this MVA have also been updated due to a change in training samples used.

  • Added mostcommonBTagIndex which returns the B meson index from which most FSP from a given reconstructed B meson candidate originated. Use case would be e.g. the FEI.

  • Added percentageMissingParticlesBTag which determines the percentage of FSP missed by using the mostcommonBTagIndex of the B meson.

  • Added percentageWrongParticlesBTag which gives the percentage of FSP reconstructed not originating from the mostcommonBTagIndex.

Modules#

1.4.6. Changes in the analysis package in light-2108-selene#

Neutral Hadron Analysis Tool#

  • Added a module that modifies 4-momentum of a neutral hadron in a given decay (e.g. \(B^0 \to J/\psi K_L^0\), \(B^0 \to p \bar{n} D^*(2010)^-\)).

  • Added a Python function reconstructDecayWithNeutralHadron, which merges the functionalities of ParticleCombiner and the above module.

Variables#

  • Removed R2EventLevel (it has been officially deprecated since release-05).

  • Modified InvM to be calculated at the decay vertex if the particle is V0 or decay at rho > 5 mm.

  • Added clusterTotalMCMatchWeight which returns the sum of all deposited MC energies that can be used to identify beam background clusters.

Modules#

  • The TrackingMomentumScaleFactors module has been extended. Now the module can scale track momenta based on phase-space dependent scaling factors given which are read from payloads according to parameters tableName and scalingFactorName. The corresponding wrapper function scaleTrackMomenta has also been extended accordingly.

  • The HelixErrorScaler has been extended to K_S0 that are reconstructed from V0. A new python function stdV0s.scaleErrorKshorts provides an alternative to the standard list K_S0:merged with a correction applied to the uncertainties of the K_S0 daughter helix parameters.

1.4.7. Changes in the framework package since release-06-00#

Conditions Database#

  • b2conditionsdb-tag-publish and b2conditionsdb-tag-invalidate have been removed. b2conditionsdb-tag-state must be used now for publishing or invalidating a globaltag.

GeneralCut rework#

  • Expanded the cut functionality by introducing a grammar based parser.

  • formula keyword is no longer necessary for writing formulas in cuts and metavariable arguments

# old
variables.addAlias('medianDzFromPV', 'extraInfo(medianDZ)')
variables.addAlias('dzFromMedianDz', 'formula(dz - medianDzFromPV)')
ma.applyCuts('pi+:fromPV', 'abs(dzFromMedianDz) < 0.05', path=main)

# new, input formula directly as metavariable argument
ma.applyCuts('pi+:fromPV', 'abs(dz - extraInfo(medianDZ)) < 0.05', path=main)

# new, variable alias without formula metavariable
variables.addAlias('dzFromMedianDz', 'dz - extraInfo(medianDZ)')
ma.applyCuts('pi+:fromPV', 'abs(dzFromMedianDz) < 0.05', path=main)

More details can be found in the Cut strings and selections section.

1.4.8. Changes in the b2bii package since release-06-00#

Conversion#

  • Added tof, sigma_tof, path_length and quality from the Mdst_tof table. They are stored in BelleTrkExtra and are not converted by default.

  • Added dedx``and ``dedx_quality from the Mdst_trk table. They are stored in BelleTrkExtra and are not converted by default.

  • Added photo_electrons and quality from Mdst_acc table. They are stored in BelleTrkExtra and are not converted by default.

Variables#

1.4.9. Changes in the mva package since release-06-00#

  • Deprecated basf2_mva_utils.calculate_roc_auc as it calculated the efficiency-purity auc not the roc (efficiency-background retention) auc. Use instead basf2_mva_utils.calculate_auc_efficiency_vs_purity or basf2_mva_utils.calculate_auc_efficiency_vs_backround_retention.

  • Added an argument overwriteExistingExtraInfo to MVAExpert and MVAMultipleExperts modules so that one can choose to overwrite or keep the old value if the given extraInfo is already defined.

  • Updated the partial_fit(...) function for python based methods. Previously the epoch parameter was confusingly defined as iEpoch * nBatches + iBatch. It now returns correctly iEpoch with a new batch argument giving the index of the current batch. Additionally added the nBatches argument to begin_fit(...) to provide the number of batches per epoch to user methods. In combination the two additions can be used to for example run the validation set after the final batch of each epoch and return the accuracy for monitoring or early stopping. For an example of this uase case see mva/examples/tensorflow/simple.py User scripts must be updated to overwrite partial_fit(state, X, S, y, w, epoch, batch) and begin_fit(state, Xtest, Stest, ytest, wtest, nBatches).

1.5. Changes since release-05#

New and updated packages in externals

Table 1.4 New and updated packages in externals from v01-09-01 to v01-10-02#

package

old version

new version

GNU Binutils

2.34

2.36.1

boost

1.72

1.75

cmake

3.16.5

3.19.7

cppzmq

4.6.0

4.7.1

curl

7.69.1

7.75.0

davix

0.7.4

0.7.6

Eigen

3.3.7

3.3.9

EOS

0.3.2

0.3.3

EPICS

7.0.3.1

7.0.5

EvtGen

R01-04-00

R02-00-01

FastJet

3.3.4

FFTW3

3.3.8

3.3.9

gcc

9.3

10.2

GDB

9.1

10.1

Geant4

10.06.p01

10.06.p03

git

2.25.2

2.31.0

git-lfs

2.10.0

2.13.2

Hammer

1.0.0

1.1.0

HDF5

1.10.6

1.12.0

HepMC

2.06.09

2.06.11

LZ4

1.9.2

1.9.3

Millepede

V04-06-00

V04-09-00

Ninja

1.10.2

nlohmann/json

3.7.3

3.9.1

pandoc

2.9.2.1

2.12

Photos

3.56

3.64

PostgreSQL

9.2.4

13.2

Python

3.6.9

3.8.8

rclone

v1.51.0

v1.54.1

ROOT

6.20/04

6.24.0

SQLite

3.31.1

3.35.1

Tauola

1.1.4

1.1.8

XRootD

4.11.2

5.1.1

zeromq

4.3.2

4.3.4

Integer Set Library

0.23

0.21

cppcheck

1.90

2.3

Doxygen

1.8.17

1.9.1

LLVM/Clang

10.0.0

12.0.0

Valgrind

3.15

3.17.0

absl-py

0.9.0

0.12.0

amply

0.1.4

anyio

3.3.0

appdirs

1.4.3

1.4.4

argon2-cffi

20.1.0

astor

0.8.1

removed

astunparse

1.6.3

async_generator

1.10

attrs

19.3.0

21.2.0

autopep8e

1.5.2

1.5.7

awkward

0.12.20

1.2.2

awkward0

0.15.5

b2luigi

0.4.4

removed

Babel

2.9.1

backcall

0.1.0

0.2.0

bleach

3.1.4

3.3.0

boost-histogram

0.7.0

1.0.2

cachetools

4.1.0

4.2.2

certifi

2020.4.5.1

2020.12.5

cffi

1.14.0

1.14.5

chardet

3.0.4

4.0.0

cheroot

8.3.0

8.5.2

cloudpickle

1.3.0

1.6.0

colorama

0.4.3

removed

ConfigArgParse

1.2.3

1.4

cryptography

2.9.2

3.4.7

cymove

1.0.0

1.0.2

Cython

0.29.15

0.29.23

decorator

4.4.2

5.0.7

defusedxml

0.6.0

0.7.1

dm-tree

0.1.6

docutils

0.15

0.17.1

filelock

3.0.12

flake8

3.7.9

3.9.2

flatbuffers

1.12

future

0.18.2

gast

0.2.2

0.3.3

gitdb

4.0.4

4.0.7

GitPython

3.1.1

3.1.14

google-auth

1.30.0

google-auth-oauthlib

0.4.4

grpcio

1.28.1

1.32.0

idna

2.9

2.10

importlib-metadata

1.6.0

removed

importlib-resources

1.4.0

removed

iniconfig

1.1.1

ipykernel

5.2.1

5.5.4

ipython

7.13.0

7.23.1

ipywidgets

7.5.1

7.6.3

jaraco.classes

3.1.0

3.2.1

jaraco.collections

3.0.0

3.3.0

jaraco.functools

3.0.0

3.3.0

jaraco.text

3.2.0

3.5.0

jedi

0.17.0

0.18.0

Jinja2

2.11.2

2.11.3

joblib

0.14.1

1.0.1

json5

0.9.6

jupyter-client

6.1.3

6.1.12

jupyter-console

6.1.0

6.4.0

jupyter-core

4.6.3

4.7.1

jupyter-server

1.10.2

jupyterlab

3.1.9

jupyterlab-pygments

0.1.2

jupyterlab-server

2.7.2

jupyterlab-widgets

1.0.0

jupytext

1.4.2

1.11.2

keras

2.3.1

2.4.3

Keras-Applications

1.0.8

removed

Keras-Preprocessing

1.1.0

1.1.2

kiwisolver

1.2.0

1.3.1

lockfile

0.12.2

removed

luigi

2.7.7

removed

lxml

4.5.0

4.6.3

Markdown

3.2.1

3.3.4

markdown-it-py

1.1.0

matplotlib

3.2.1

3.4.2

matplotlib-inline

0.1.2

mdit-py-plugins

0.2.8

metakernel

0.24.4

0.27.5

more-itertools

8.2.0

8.7.0

mysql-connector-python

8.0.24

nbclassic

0.3.1

nbclient

0.5.3

nbconvert

5.6.1

6.0.7

nbformat

5.0.6

5.1.3

nbsphinx

0.6.1

0.8.4

nest-asyncio

1.5.1

notebook

6.0.3

6.4.3

numexpr

2.7.1

2.7.3

numpy

1.18.3

1.19.5

opt-einsum

3.2.1

3.3.0

packaging

20.3

20.9

pandas

1.0.3

1.2.4

pandocfilters

1.4.2

1.4.3

parse

1.15.0

removed

parso

0.7.0

0.8.2

pbr

5.4.5

5.6.0

Pillow

7.1.1

8.2.0

pip

20.0.2

21.1.1

pluggy

0.13.1

portend

2.6

2.7.1

prometheus-client

0.7.1

0.10.1

prompt-toolkit

3.0.5

3.0.18

protobuf

3.11.3

3.16.0

psutil

5.7.0

5.8.0

ptyprocess

0.6.0

0.7.0

PuLP

2.4

py

1.10.0

pyasn1

0.4.8

pyasn1-modules

0.2.8

pycodestyle

2.5.0

2.7.0

pydocstyle

5.0.2

6.0.0

pyflakes

2.1.1

2.3.1

Pygments

2.6.1

2.9.0

PyJWT

1.7.1

2.1.0

pyroofit

0.9.4

0.9.6

pyrsistent

0.16.0

0.17.3

pytest

6.2.4

python-daemon

2.2.4

removed

pytz

2019.3

2021.1

PyYAML

5.3.1

5.4.1

pyzmq

19.0.0

22.0.3

qtconsole

4.7.3

5.1.0

requests

2.23.0

2.25.1

requests-unixsocket

0.2.0

rise

5.6.1

5.7.1

rsa

4.7.2

scikit-learn

0.22.2.post1

0.24.2

scipy

1.4.1

1.6.3

SCons

3.1.2

4.1.0.post1

seaborn

0.10.0

0.11.1

setuptools

46.1.3

56.2.0

six

1.14.0

1.15.0

sly

0.4

smart-open

5.0.0

smmap

3.0.2

4.0.0

snakemake

5.15.0

6.3.0

sniffio

1.2.0

snowballstemmer

2.0.0

2.1.0

Sphinx

3.0.2

4.0.0

stopithread

1.1.2

tabulate

0.8.7

0.8.9

tempora

3.0.0

4.0.2

tensorboard

1.15.0

2.5.0

tensorboard-data-server

0.6.1

tensorboard-plugin-wit

1.8.0

tensorflow

1.15.2

2.4.1

tensorflow-estimator

1.15.1

2.4.0

tensorflow-probability

0.9.0

0.12.2

terminado

0.8.3

0.9.4

Theano

1.0.4

1.0.5

threadpoolctl

2.1.0

toml

0.10.2

toposort

1.5

1.6

torch

1.4.0

1.8.1

torchaudio

0.8.1

torchvision

0.5.0

0.9.1

tornado

5.1.1

6.1

tqdm

4.60.0

traitlets

4.3.3

5.0.5

typing-extensions

3.7.4.3

uncertainties

3.1.2

3.1.5

uproot

4.0.7

uproot3

3.11.3

3.14.4

uproot3-methods

0.7.3

0.10.1

urllib3

1.25.9

1.26.4

wcwidth

0.1.9

0.2.5

websocket-client

1.2.1

wheel

0.34.2

0.36.2

zipp

3.1.0

removed

zstandard

0.15.2

Removal of old and deprecated database functions

Some functions used in the past to handle the conditions database (like basf2.use_local_database or basf2.reset_database) are removed, and any script using them does not work anymore. This removal does not imply any functionality loss, since the users can use the basf2.conditions object to properly configure the conditions database in their steering files (see also Configuring the Conditions Database).

Photons generated by PHOTOS in continuum events

Fixed the issue where PHOTOS photons were not correctly flagged in continuum events, e.g., charm decays (5828). This was present in release-05-00-01 and earlier, including MC13 files.

Unification of B2BII settings

A single switch between Belle and Belle II settings has been implemented, which is automatically set when reading in a Belle type mdst. No individual options have to be set in modular analysis functions.

1.5.1. Changes in the analysis package since release-05-02#

Vertex Fitting#

  • KFit has been extended to be able to handle vertex fits (with and without mass constraint) that involve an eta particle reconstructed in eta -> gamma gamma.

  • Added a method scaleError in order to scale up helix errors and avoid underestimation of vertex error.

  • Added a new fit-type option massfourC to function vertex.kFit(). The kinematic fit is performed with the 4 momentum constraint of a mother particle and mass constraints of intermediate particles specified by the option massConstraint, simultaneously.

  • Changed the treatment of Bremsstrahlungs-corrected tracks in vertex.treeFit. The previous implementation exhibited bad performance when a Bremsstrahlungs-photon is applied. The new implementation fits the corrected Track as a single particle, analogous to, what KFit does.

Variables#

Modules#

  • Added a module TrackIsoCalculator, which takes as input a standard charged particle list, and calculates for every particle’s track in the list the 3D distance to the closest track in the event. The distance is calculated as the 3D separation between the points where two extrapolated track helices cross a given detector’s inner surface, where the detector is an input parameter to the module. This variable can be used to parametrise charged particle identification efficiencies and mis-identification probabilities, as they can depend on the activity around each charged particle candidate.

  • The InclusiveDstarReconstruction creates antiparticle lists correctly now. The module’s input changed to a DecayString of the form D* -> pi and MC Matching is applicable (one can use isSignal).

  • In BtubeCreator module, functionality to apply a cut on the confidence level of the fit of fully reconstructed B to the beamspot is added.

  • The EventKinematics module can now compute event kinematics using generated particles as an input.

  • Arguments of writePi0EtaVeto have been updated. downloadFlag and workingDirectory have been removed since the download processes can be skipped. New arguments have been added for several reasons. suffix allows to calculate this veto for multiple photons. hardParticle allows to call this function for a given particle other than a photon. Four new arguments have been added to override the payload names and soft photon selections, pi0PayloadNameOverride, etaPayloadNameOverride, pi0SoftPhotonCutOverride, and etaSoftPhotonCutOverride.

  • A new helper method updateROEUsingV0Lists has been added to facilitate application of V0Finder results in Rest Of Event.

  • Added a possibility to add a ROE mask for KLM-based particles and an experimental option of including KLM-based particles into ROE 4-vector computation.

    Warning

    The option useKLMEnergy of RestOfEventBuilder module is only meant for performance studies and NOT for a physics analysis.

  • The PrintMCParticles module and thus also the printMCParticles function has a completely new layout that should be much easier to parse, especially for complicated events. By default it shows much less information but in an easier to parse tree representation. See the documentation of printMCParticles for details.

  • The ParticleLoader now creates photon candidates from KLMCluster if the parameter loadPhotonsFromKLM is set to true. It is off by default.

    Warning

    Photons from KLMCluster should only be used in specific use-cases and after a thorough study of their effect.

  • In BtubeCreator, new extrainfo TubeB_p_estimated was added. This returns the magnitude of the estimated momentum of the B which should fly in the direction of the Btube.

  • Added a module HelixErrorScaler, which multiplies constant scaling factors to helix errors of input charged particles and stores them in a new list.

  • In ParticleStats add functionality to produce a json containing the information printed on stdout. Added also a tool b2plot-particleStats: Analyze a json produced by ParticleStats to produce plot of retention rate and pass matrix to analyze the json and produce some plots.

  • The TrackingMomentumScaleFactors module has been extended. A ParticleList of composite particles can now be processed as well. In that case the momenta of all track-based daughter particles are scaled by the provided factor.

  • The wrapper functions for the tracking systematics modules have been renamed from trackingMomentum to scaleTrackMomenta and from trackingEfficiency to removeTracksForTrackingEfficiencyCalculation.

  • Added the TauDecayMode module, which is an actualization of the TauDecayMarker module for the new TauolaBelle decays. Using a txt file, which defines the mapping between decay strings and decay numbers from TauolaBelle, the module assigns a decay number for each tau in the event. This decay number is stored in the variables tauMinusMCMode and tauPlusMCMode. It’s possible to provide the path of a different txt file for the mapping as a parameter to the module.

  • Added a EnergyBiasCorrection module, sub-percent correction only applied to E of photons not on clusterE and should only be applied on data. correctEnergyBias is a corresponding wrapper function.

  • The ChargedPidMVA and ChargedPidMVAMulticlass modules now apply by default charge-dependent BDT training for particle identification. The charge-independent training can be used optionally.

  • Added a PhotonEfficiencySystematics module which adds photon detection efficiency Data/MC ratios, systematic, statistical and total uncertainties as variables in extrainfo to a given photon list. Ratios can only added for particles in photon lists. addPhotonEfficiencyRatioVariables is a corresponding wrapper function.

  • Bug fixed in FlavorTaggerInfoBuilder and FlavorTaggerInfoFiller: FlavorTaggerInfoMap is created now in the builder to make it accessible in main paths outside the ROE loops. This permits now to save the FT outputs for every B-meson candidate.

Full Event Interpretation#

  • Added option to reconstruct strange B mesons (at Y(5S)) in 51 decay channels. Can be switched on with the strangeB flag in fei.get_default_channels().

  • The FEI has been retrained with MC14 and release-05. The prefix FEIv4_2021_MC14_release_05_01_12 has to be set in the FEI configuration.

Standard Particle Lists#

  • The selection criteria of the pi0 standard particle lists have been updated from the January 2020 to the May 2020 recommendations.

  • Defined new stdCharged.stdE and stdCharged.stdMu lepton lists, based on uniform target efficiency selection working points. Available both for likelihood-ratio based PID selection and new BDT-based selection, both global and binary (\(\ell\) vs. \(\pi\)). These are recommended for analysis once correction factors payloads become available.

1.5.2. Changes in the analysis package since release-05-01#

Variables#

  • Added b2help-variables which behaves identical to basf2 variables.py but more in-keeping with the b2help-<something> theme.

Full Event Interpretation#

  • Background sampling for \(B^{0} \rightarrow J/\psi K_{S}^{0}\) and \(B^{+} \rightarrow J/\psi K^{+}\) are deactivated.

  • The baryonic tagging modes are activated by default.

1.5.3. Changes in the analysis package since release-05-00#

Bremsstrahlung correction#

  • Fixed that the copies of particles to which Bremsstrahlung was applied (whether this resulted in a added photon or not) are considered as sources of tracks. This allows to update the daughters in vertex fits that involve brems-corrected particles.

Variables#

Modules#

  • Added the new wrapper function applyRandomCandidateSelection, which uses the BestCandidateSelection module to reduce the number of candidates in the input particle list to one candidate per event based on a random value.

  • In TagVertex, all charged particles from the ROE are loaded to be included in the tag vertex fit (and not only those with a pion hypothesis)

  • The ParticleCombinerFromMC and the reconstructMCDecay function can set decayModeID extraInfo with an argument dmID. One can decide whether charge conjugated mode should be reconstructed with new boolean argument chargeConjugation (true by default).

  • The special treatment of MCMatching for the tau-decay is fixed. The treatment was working fine in release-04, but it was broken in the release-05.

1.5.4. Changes in the framework package since release-05-01#

  • Added b2help-modules which behaves identical to basf2 -m but more in-keeping with the b2help-<something> theme.

Conditions Database#

  • The format in which local database payloads are created has changed slightly, see Creation of new payloads

  • b2conditionsdb-tag-merge has been added to merge a number of globaltags into a single globaltag in the order they are given. The result is equivalent to having multiple globaltags setup in the conditions access for basf2.

  • b2conditionsdb-tag-runningupdate has been added to calculate and apply the necessary updates to a running globaltag with a given staging globaltag.

  • b2conditionsdb-download has learned a new argument to clean all payload files not mentioned in the database file from the download area.

Tools#

  • Added b2rundb-query to perform simple rundb queries from the command line.

Python Interface#

  • The ANSIColors class has been added to the module terminal_utils to very easily add color output to the terminal.

  • Added a RunDB class which makes writing scripts using the rundb more straightforward.

1.6. Changes since release-04#

Neutral hadrons from ECLClusters get momentum from the cluster energy

Since release-04 it has been possible to load ECLClusters under the neutral hadron hypothesis. Previously we assumed a mass when calculating the particle momentum, however this leads to problems when, for example, a \(K_L^0\) deposits less than its mass energy in the ECL. This happens about 50% of the time.

The momentum of neutral hadrons from the ECL is now set to the clusterE.

Bremsstrahlung correction

The BremsFinder module has been developed to find relations between tracks and photons that are likely to have been emitted by these tracks via Bremsstrahlung. The matching quality figure of merit is based on the angular distance between the photon ECL cluster and the extrapolated hit position of the track at the ECL. The function correctBrems performs the actual correction. There is also a reimplementation of Belle’s Bremsstrahlung correction approach of looking for photons in a cone around tracks (correctBremsBelle), which is recommended for b2bii analyses.

Warning

While it is technically possible to perform a TreeFit after applying Bremsstrahlung correction, the fit performance is unfortunately quite bad. However, there is already an improvement in the pipeline that should fix this issue. It will probably be available in one of the next light releases.

MC reconstruction and MC matching

The ParticleCombinerFromMC module and its corresponding wrapper function reconstructMCDecay should be used instead of findMCDecay to reconstruct decay modes based on MC information.

The DecayStringGrammar has been extended with new exception markers for Bremsstrahlung, decay in flight, and misidentification.

Exceptions for the MC matching of daughter particles with the DecayStringGrammar are propagated to the mother particle.

Redefinition of angle variables

The kinematic variables decayAngle, daughterAngle and pointingAngle now return the angle instead of its cosine.

Protection of ParticleLists and particle combinations

It is no longer allowed to use the label "all" for a particle list if a cut is applied. Reconstructed decays need to preserve electric charge. However, this can be deactivated if you know what you are doing, e.g. in searches for New Physics.

1.6.1. Changes in the analysis package since release-04-02#

Warning

Global includes like from basf2 import * are almost completely removed from the analysis package.

Vertex Fitting#

Warning

The convenience function for the TreeFitter module has been renamed from vertexTree to vertex.treeFit().

Warning

The KFit convenience functions have been merged to the new function vertex.kFit(). One of its input arguments is fit_type, which specifies whether you want to perform a mass fit (fit_type=mass), a vertex fit (fit_type=vertex), a mass-constrained vertex fit (fit_type=massvertex), or a vertex fit with the mother constrained to the beam four-momentum (fit_type=fourC).

  • Added smearing parameter to vertex.kFit. When you perform a vertex fit with an IP tube constraint (fit_type=vertex and constraint=iptube), the IP profile is smeared by the given value. The IP tube is defined by a neutral track pointing towards the boost direction starting from the smeared IP profile. Setting smearing=0 gives the original iptube option, which is an elongated ellipsoid in boost direction.

  • Fixed TreeFitter bias on V0s by propagating the correct TrackFitResult from V0 daughters (see 6220).

  • Added FeedthroughParticle and InternalTrack to vertex.treeFit. This now provides compatibility of TreeFitter with the bremsstrahlung recovery modules. In principle, TreeFitter can now be used for decays for which Bremsstrahlung correction has been applied. However, there are massive performance issues. Do not blindly use TreeFit but cross-check your results with KFit. Further improvements are planned.

Standard Particle Lists#

  • The list label "all" is now forbidden from use with cuts. This can introduce some very subtle bugs in user scripts. The following code will no longer work:

    fillParticleList("gamma:all", "clusterE > 0.02", path=mypath)

    Instead you should replace it with a meaningful name of your choice. For example:

    fillParticleList("gamma:myMinimumThresholdList", "clusterE > 0.02", path=mypath)

  • Added stdHyperons.py, this analysis script reconstructs the standard hyperons Xi-, Xi0, and Omega-.

  • The default vertex fitter for the V0 lists stdKshorts and stdLambdas has been changed from Rave to TreeFit. However, a new option called fitter has been added that allows to change the vertex fitter back to raveFit or to kFit. This change is motivated by a much faster execution time (see 5606) and a compatible performance as summarized here.

  • Added a new standard list stdCharged.stdMostLikely which creates 5 mutually-exclusive lists of track-based particles under their most likely hypothesis.

  • Moved the KlId check from ParticleLoader to stdKlongs. This modification does not change the definition of the K_L0:all list and, in general, of the user-defined \(K_{L}^0\) lists.

  • Removed the functions mergedKshorts and mergedLambdas, which returned the standard V0 lists anyway.

  • Removed the deprecated pi0 lists pi0:loose and pi0:veryloose

  • stdPi0s and stdPhotons updated to match Jan2020 recommendations. Naming scheme updated to indicate date of optimization.

Variables#

Note

All variables will return a quietNaN instead of -999 or other out-of-the-sky numbers in case of errors. To study the failures use the meta variable ifNANgiveX, which replaces nan with a value of your choice.

Warning

The variables decayAngle, daughterAngle and pointingAngle now return an angle instead of a cosine.

Note

This is not the generator-level match but simply ignores photons added by the BremsFinder.

Hint

It is recommended to use variablesToExtraInfo prior to a vertex fit to access pre-fit values, not only for mass significances but also for vertex positions, etc.

  • Fixed truth kinematic variables (mcPX, mcPY, …) to respect the current reference frame, so that they can be accessed not only in the lab frame but for example as well in the CMS frame using useCMSFrame.

  • Extended functionality of recoil kinematics variables (pxRecoil, pyRecoil, …) to be usable with any requested frame.

  • Added isFromV0 to identify particles that come from tracking-level V0’s (as opposed to those created at analysis-level using modularAnalysis.reconstructDecay).

  • Removed isExtendedSignal whose exact behavior is mirrored by isSignalAcceptWrongFSPs.

  • Added the new variable collection event_kinematics which contains all variables available after running the EventKinematics module.

  • Added unmask, which unmasks certain indicated bits in a variable (set certain bits in a variable to zero).

  • Added a new variable util create_isSignal_alias which creates customized isSignal, accepting certain match errors.

  • Removed the previously deprecated variable clusterHypothesis.

  • Added firstCDCLayer, returns the ID of the CDC layer closest to the IP with a hit attributed to the particles track

  • Added the new variable collection cluster_average. It contains the cluster variables that can only be executed for particles with daughters, currently maxWeightedDistanceFromAverageECLTime and weightedAverageECLTime.

  • Added a variable for TDCPV resolution study DeltaTRes and legacy Belle DeltaTBelle.

  • Added genIPX, genIPY, and genIPZ; the generator-level IP.

  • Clarified the documentation of the measured interaction point variables (IPX ff).

Modules#

  • When merging ParticleLists using the ParticleListManipulator (modularAnalysis.copyLists) duplicates (not multiple candidates) are removed on a first-come, first-served basis, i.e. the order of the input particles matters. Now, the new parameter variable can be provided to determine which duplicate should be preferred. The logic is the same as for the BestCandidateSelection module. Furthermore, duplicates, which differ only by the order of their daughter particles, are now identified.

  • The ParticleLoader can create neutron particle lists now. ECL cluster with the hadron hypothesis as well as KLM cluster are used as source.

  • Tweaks to the ParticleLoader for how it creates particles from V0s, it now preserves the link to the mdst object. These changes are all under-the-hood and require no actions by users.

  • In TreeFitter bugfix in mass constraint: chi2 correctly calculated now.

  • In TreeFitter change of initialisation values increases performance in all fits with tracks.

  • In TreeFitter max iterations set to 100, increases efficiency slightly in many cases.

  • Removed module CreateBsigFromFullRecoBtag

  • In BtubeCreator module, functionality to associate Btube to user’s chosen B is added.

  • Removed module FSRCorrection in favour of BremsFinder and BelleBremRecovery.

  • In ChargedPidMVA module, remove handling of particle list modified by the removed FSRCorrection module.

  • In ChargedPidMVA module, apply MVA category cuts retrieved from the DB payload.

  • Added ChargedPidMVAMulticlass module to allow performing global PID (one-vs-rest), using an MVA trained in multi-class mode.

  • Added a BremsFinder to unpack the bremsstrahlung relations provided by tracking.

Hint

This tool is the recommended way to correct for bremsstrahlung photons in Belle II analyses.

  • Modified writePi0EtaVeto function and added oldwritePi0EtaVeto function. The weight files used in release-03 were not available in release-04. The latter uses old weight files optimized with MC9, while new weight files, which are optimized with MC12, are used in writePi0EtaVeto

  • Enabled to select daughters which will be used to perform fitting with KFit. One can use the selector ^ to select daughters.

  • KFit can be used to fit vertices of fully-inclusive particles, for which it ignores the daughters without defined p-value.

  • RestOfEventBuilder, EventKinematics and EventShapeCalculator modules can use most likely charged particle hypothesis according to PID information.

  • Added the ParticleCombinerFromMC module and the reconstructMCDecay to find and create ParticleList from a given DecayString. They can be used instead of the MCDecayFinder and findMCDecay which are not fully tested and maintained.

  • In TagVertex added Btube as a possible constraint for the tag B vertex fit. The useFitAlgorithm parameter in the module is replaced by constraintType (which can be set to tube) and trackFindingType. All computations done in the module are now using double precision. The module is also updated to allow for the computations of new variables related to the tracks used in the tag vertex fit.

  • Added a InclusiveDstarReconstruction to inclusively reconstruct D* mesons by estimating the four vector using slow pions.

  • In the TagV function, added the possibility to use KFit instead of Rave to perform the tag vertex fit.

  • Both flavor taggers support ROE masks. Added new maskName argument.

  • Added new DecayString grammar for c_AddedRecoBremsPhoton, c_DecayInFlight, and c_MisID. If one uses ?addbrems in own DecayString, c_AddedRecoBremsPhoton is ignored, thus isSignal works as isSignalAcceptBremsPhotons. One can add (decay) and/or (misID) to the beginning of the particle name to accept c_DecayInFlight and/or c_MisID. The following is an example, 'reconstructDecay('D0:sig -> (misID)K-:loose (decay)pi+:loose', '', path=mypath)'

  • Modified modularAnalysis.matchMCTruth to work always recursively. Once one calls modularAnalysis.matchMCTruth for a particle, the MC Matching is done for not only the particle but also all daughters of the particle correctly. This modification does not change the MC Matching of the particle for which modularAnalysis.matchMCTruth is called.

  • Added feature to add multiple TTrees to the same output file via the VariablesToEventBasedTree module.

  • Rest Of Event can be created using MCParticles using modularAnalysis.buildRestOfEventFromMC

  • Removed -->, =>, and ==> from the list of allowed arrow types for the DecayString.

  • In TagVertex module, added the possibility to use the truth information from the tag particles in the vertex fit. The option useTruthInFit = True switches that on.

  • In TagVertex module, implemented an internal change: tag particles are loaded as Particle object from the ROE and not anymore as TrackFitResult. This should have no effect to users.

  • Added a warning to the ParticleCombiner module for decay strings violating electric charge conservation which can be turned off by setting allowChargeViolation=True.

  • In TagVertex module, added the possibility to perform vertex fit with the tag particle tracks rolled back to their primary vertex points. The option useRollBack = True switches that on.

  • Added an argument to the ParticleCombiner module that allows to deactivate the automatic reconstruction of the charge-conjugated mode. In reconstructDecay the option is called chargeConjugation, which is True by default.

  • The MC matching with the DecayString properly works in hierarchical decay. For example,

    from modularAnalysis import reconstructDecay, matchMCTruth
reconstructDecay('B0:signal      -> mu+ mu- ... ?gamma',   '', path=mypath)
reconstructDecay('B0:generic     -> D*-:Dpi pi+:all',      '', path=mypath)
reconstructDecay('Upsilon(4S):BB -> B0:generic B0:signal', '', path=mypath)
matchMCTruth('Upsilon(4S)', path=mypath)

    In the above case, missing daughters (massive FSP and gamma) of B0:signal are accepted for not only B0:signal but also Upsilon(4S), so that isSignal can be 1. Another example,

    reconstructDecay('D-:pipi0 -> pi-:all pi0:all',      '', path=mypath)
reconstructDecay('B0:Dpi   -> D-:pipi0 pi+:all ...', '', path=mypath)
matchMCTruth('B0:Dpi', path=mypath)

    In this case, one wants to accept missing massive daughters in B0:Dpi decay but not in D-:pipi0 decay. So, if the decay of D-:pipi0 in the MC truth level is D- -> pi+ pi- pi- pi0, isSignal of D-:pipi0 and B0:Dpi will be 0, since there are missing daughters in D-:pipi0 decay. If one wants to accept missing daughters in D-:pipi0, please use the DecayString grammar in the reconstruction of D-:pipi0 or use isSignalAcceptMissing variable instead of isSignal.

Utilities and core objects#

  • A set of functions DistanceTools has been added to compute distances between (straight) tracks and vertices.

  • Added a relation between Track-based Particles and TrackFitResults.

  • Added a getTrackFitResult method to Particle.

  • Added a function to retrieve electric charge of a particle based on its pdg code.

  • Added a RotationTools.h file with few functions related to rotation

Tutorials and Examples#

  • Added a tutorial about creating aliases (examples/VariableManager/variableAliases.py)

1.6.2. Changes in the analysis package since release-04-00#

Variables#

  • Now vertex variables dr, dx, dy, dz, dphi, dcosTheta... take into account the nontrivial transformation of track parameters relative to IP when used for tracks. dr did not center at zero before and now it does.

  • Added isDescendantOfList and isMCDescendantOfList meta variables that allow to check whether the particle is the descendant of the list (or if it is matched to the descendant of the list) at any generation. The variables have recursive search inside, they extend isDaughterOfList and isGrandDaughterOfList.

  • Added mcParticleIsInMCList which checks the underlying MC particles (either matched or generator level).

  • Fixed bug and renamed mcFlavorOfOtherB0 variable to mcFlavorOfOtherB, which accepts now neutral and charged B candidates.

  • Removed clusterCRID which duplicates clusterConnectedRegionID.

  • Fixed goodBelleLambda and now it returns extraInfo(goodLambda) on Belle data.

  • Allow use of meta variables in creation of aliases by replacing non alphanumeric characters with underscores in the alias name

  • Modified daughterAngle to accept generalized variable indices instead of simple indices. A generalized index is a column-separated string of daughter indexes belonging to different generations, starting from the root particle. For example, 0:2:1 indicates the second daughter (0) of the third daughter (2) of the first daughter (0) of the particle. Of course, conventional indexes are still working as expected: 1 still indicates the second daughter of the particle.

  • Added daughterCombination, that returns a variable calculated on the sum of an arbitrary number of daughters. Generalized indexes are supported. This variable is mostly intended to calculate the invariant mass or the recoil mass of any set of particles belonging to a decay tree.

  • Fixed isSignal, which always accepted c_MissingResonance and c_MissFSR/c_MissPHOTOS even if one used =direct=>, =norad=>, or =exact=>. Now it correctly respects the decay string grammar. The other isSignal* variables, such as isSignalAcceptMissing, are also fixed.

  • Added useAlternativeDaughterHypothesis, that returns a variable re-calculated using alternative mass assumptions for the particle’s daughters.

  • Restructured the mc_flight_info collection by removing the non-sense error variables and replacing mc_flightTime and mc_flightDistance, which were aliases created using matchedMC, with the dedicated variables mcFlightTime and mcFlightDistance, respectively.

  • Removed output_variable option in the Deep Flavor Tagger and introduced a standard variable DNN_qrCombined for the output. The new variable returns the flavor times the dilution factor as the category-based Flavor Tagger output variables FBDT_qrCombined and FANN_qrCombined do. Now we can evaluate the output of both taggers in the same way.

  • Bug fix to guard against range exceptions in KSFWVariables (see 6026).

Modules#

  • In TreeFitter fixed a bug in life time calculation. A constant in the jacobian was missing. As a result the propagated error was slightly overestimated.

  • Fix to nested RestOfEvent objects (see 5560)

  • Fixed bugs in MCDecayFinder and findMCDecay. Inefficiency and large background when one used =direct=> or sub-decay such as D*+ -> [D0 -> pi+ pi- pi0] pi+ are fixed. But still the module has some bugs if one uses K_S0.

Conditions DB#

  • In ChargedPidMVAWeights payload class, added MVA category cut strings in basf2-compliant format.

Full Event Interpretation#

  • Addition of hadronic FEI channels involving baryons. This includes the addition to default channels the following particles: p, Lambda_c+, Sigma+ and Lambda0. Baryonic modes must be switched on when calling particles = fei.get_default_channels(baryonic = True) using a flag (baryonic = True).

1.6.3. Changes in the framework package since release-04-00-00#

Conditions Database#

  • b2conditionsdb: Conditions DB interface has been optimized to work with larger globaltags.

  • b2conditionsdb-diff and b2conditionsdb-iov by default don’t show any database internal ids anymore but those can be re-enabled with --show-ids

  • b2conditionsdb-dump has learned a new argument to show the content of a payload valid for a given run in a given globaltag

  • There are new python classes to handle iovs, see conditions_db.iov

Logging#

Miscellaneous#

  • Added support for writing udst files as output from BGxN N>0 files in light releases (see BII-3622). This means skimming is fully supported with a light release.

  • Added support for b2bii in light releases. However, this comes at the cost of no longer being able to convert ExtHits and ECLHits.

  • The RootInput will now by default skip events which have an error flag set. This can be changed with the discardErrorEvents parameter.

  • Added the function basf2.get_file_metadata to quickly obtain the FileMetaData object of a given basf2 output file.

  • Added the tools b2code-sphinx-build and b2code-sphinx-warnings to build the sphinx documentation or check for warnings when building the documentation.

1.6.4. Changes in the b2bii package since release-04-01#

Warning

b2biiConversion.setupB2BIIDatabase() will be deprecated soon.

convertBelleMdstToBelleIIMdst#

  • Added a switch to deactivate nisKsFinder.

  • Renamed applyHadronBJSkim to applySkim, and added switches to deactivate HadronA and HadronB skims.

  • Added a switch to deactivate converting RecTrg_Summary3 table.

Conversion#

  • Removed convertECLCrystalEnergies() and convertExtHits() in the conversion.

  • Added mass-constraint fit information of pi0:mdst, including chiSquared, ndf and pValue as extraInfo().

  • Added m_final(3) in RecTrg_Summary3 table as eventExtraInfo(rectrg_summary3_m_final).

1.7. Changes since release-03#

Removal of default analysis path and NtupleTools

Warning

The default path (”analysis_main”) and the NtupleTools are now removed.

This is a major backward-compatibility breaking change. Please update your user scripts to create your own path (basf2.create_path) and to use the variable manager tools (such as VariablesToNtuple).

If your previously working example script from release-03 looked something like this:

from basf2 import *
from stdCharged import stdPi
from modularAnalysis import *
stdPi("good")
ntupleFile("myFile.root") # <-- now removed
ntupleTree("pi+:good", ['pi+', 'Momentum']) # <-- now removed
process(analysis_main)
print(statistics)

You should update it to this:

import basf2 # better not to import all
from stdCharged import stdPi
from modularAnalysis import variablesToNtuple
mypath = basf2.Path() # create your own path (call it what you like)
stdPi("good", path=mypath)
variablesToNtuple("pi+:good", ['px', 'py', 'pz', 'E'], path=mypath)
basf2.process(mypath)
print(basf2.statistics)

See also

The example scripts available here:

$BELLE2_RELEASE_DIR/analysis/examples/VariableManager

Switch of beam spot information from nominal to measured values.

The interaction point position and its uncertainties are now taken from the database with values provided by the tracking group. All beam kinematics information is also moved to the database, which will eventually be measured on data. For now they are the values provided by the accelerator.

Warning

The previous definition of the boost included a small rotation to align it with the HER. This is no longer possible with the new structure. The definition of CMS is therefore slightly changed. The impact should be at the percent level.

If you have a physics analysis sensitive to this change: please discuss with the software / performance groups and add a comment to 4294.

See also

The beam information can be accessed with Ecms, beamPx, beamPy, beamPz, and beamE.

Note

As a consequence of this, there is now a difference between the variables ( dx, dy, dz ) compared to ( x, y, z ); and similarly for variables ( mcDecayVertexX, mcDecayVertexY, mcDecayVertexZ ) compared to ( mcDecayVertexFromIPX, mcDecayVertexFromIPY and mcDecayVertexFromIPZ ).

Redesign of the Conditions Database Interface

The configuration and handling of the connection to the conditions database has been completely rewritten in a more coherent and modular way. We now have a new and consistent configuration interface, global tag replay and advanced checks: If users specify a global tag to be used which is either marked as invalid in the database or which cannot be found in the database the processing is now aborted. See Conditions Database Overview for details.

Restrict usage of useDB=False for Geometry creation

Creating the geometry from XML files instead of the configuration in the Database may lead to wrong results. So while the option useDB=False is still necessary to debug changes to the geometry definitions it is now restricted to only be used for exp, run = 0, 0 to protect users from mistakes.

This also changes the behavior of add_simulation() and add_reconstruction(): If a list of components is provided this will now only change the digitization or reconstruction setup but will always use the full geometry from the database.

Loading ECLClusters under multiple hypotheses

It is now possible to load \(K_L^0\) particles from clusters in the ECL. This has several important consequences for the creation of particles and using combinations containing \(K_L^0\) s or other neutral hadrons in the analysis package. This is handled correctly by the ParticleLoader and ParticleCombiner (the corresponding convenience functions are modularAnalysis.fillParticleList and modularAnalysis.reconstructDecay). Essentially: it is forbidden from now onwards for any other analysis modules to create particles.

Deprecated RAVE for analysis use

The (external) RAVE vertex fitter is not maintained. Its use in analysis is therefore deprecated. We do not expect to remove it, but do not recommend its use for any real physics analyses other than benchmarking or legacy studies.

Instead we recommend you use either KFit (vertex.kFit) for fast/simple fits, or TreeFit (vertex.treeFit) for more complex fits and fitting the full decay chain. Please check the Tree Fitter pages for details about the constraints available. If you are unable to use TreeFitter because of missing functionality, please submit a feature request!

Warning

The default fitter for vertex.fitVertex has been changed to KFit.

Tidy up and rename of Helicity variables.

Renamed helicity variables in the VariableManager following consistent logic. We added the new variable cosAcoplanarityAngle.

Warning

cosHelicityAngle is now cosHelicityAngleMomentum, and cosHelicityAngle has a new definition (as in the PDG 2018, p. 722).

Old name

New name

cosHelicityAngle

cosHelicityAngleMomentum

cosHelicityAnglePi0Dalitz

cosHelicityAngleMomentumPi0Dalitz

cosHelicityAngleIfCMSIsTheMother

cosHelicityAngleBeamMomentum

New DecayStringGrammar for custom MCMatching

Users can use new DecayStringGrammar to set properties of the MCMatching. Then isSignal, mcErrors and other MCTruthVariables behave according to the property.

Once DecayStringGrammar is used with reconstructDecay, users can use isSignal instead of several specific variables such as isSignalAcceptMissingNeutrino. If one doesn’t use any new DecayStringGrammar, all MCTruthVariables work same as before.

The grammar is useful to analyze inclusive processes with both fully-inclusive-method and sum-of-exclusive-method. There are also new helper functions genNMissingDaughter and genNStepsToDaughter to obtain the detailed MC information.

You can find examples of usage in Marker of unspecified particle, Grammar for custom MCMatching.

1.7.1. Changes in the analysis package#

TreeFitter#

  • Fix the \(\phi\)-dependent loss of performance for displaced vertices (4674).

Flavor Tagger#

  • Default Expert (testing mode) does not create repositories and does not save weight files locally. It only loads the payloads directly from the database using the database payload names as mva identifiers.

  • BtagToWBosonVariables adapted to work with the new ROE.

  • All release validation and performance evaluation scripts added to $BELLE2_RELEASE_DIR/analysis/release-validation/CPVTools.

  • The flavor tagger creates and adds default aliases into the collection list flavor_tagging.

Vertex Fitting#

  • Added IP tube constraint to KFit.

  • The parameter confidenceLevel of the ParticleVertexFitter now always rejects the particle candidates with a p-value lower than the specified one. Specifically, setting confidenceLevel to 0 does not reject candidates with p-value equal to 0 anymore. Thus, the meaning of this parameter is now the same as for the TreeFitter.

FEI#

  • Removed the backward compatibility layer (pid_renaming_oktober_2017). Only FEI trainings from release-02 are supported. Please update to FEIv4_2018_MC9_release_02_00_01 or newer.

Variables#

Warning

we overhauled the helicity variables and added new ones to replace the NtupleHelicityTool. We renamed cosHelicityAngle to cosHelicityAngleMomentum, cosHelicityAnglePi0Dalitz to cosHelicityAngleMomentumPi0Dalitz, and cosHelicityAngleIfCMSIsTheMother to cosHelicityAngleBeamMomentum. We added the variables cosHelicityAngle and cosAcoplanarityAngle defining them as the PDG 2018, p. 722.

  • The function to create variable aliases for particles selected from a decay string has been fixed and extended to allow better customization (4368)

    See also

    variables.utils.create_aliases_for_selected()

  • Added a variable that returns the distance between a KLMCluster and its associated Track klmClusterTrackDistance

  • Access to tag vertexing quality indicators for resolution function study, TagVNDF, TagVChi2, and TagVChi2IP.

  • Statistically correct combination of p-values pValueCombination

  • Added the meta variable clusterHasFailedTiming to check if the ECL cluster’s timing fit failed.

  • Added the meta variable clusterHasFailedErrorTiming to check if the calculation of the ECL cluster’s error on the timing failed.

  • The chi2 and ndf from KFit vertices are stored as extraInfo for debugging.

  • Added trackFitHypothesisPDG which returns the PDG code of the successful track fit hypothesis that was used to create this particle (that might be different from the PDG code of the particle list)

  • Added variables for 2D Armenteros-Podolanski plot, ArmenterosLongitudinalMomentumAsymmetry, ArmenterosDaughter1Qt, and ArmenterosDaughter2Qt

  • Fall-back value changed from zero to NaN for several track variables.

  • Added sourceObjectIsInList which checks whether there are particles from the same underlying track or cluster in a given particle list. Similar to isInList but a tighter check.

  • Added isCloneOfSignalSide and hasAncestorFromSignalSide variables to monitor the clone particle contamination in ROE. These variables use generator information.

  • The variables for the fast reco HLT trigger stage have been removed, as we are currently not planning to have it in the early data taking.

  • The event-based variable KLMEnergy is removed. It is completely meaningless (4919).

  • The variables ( dx, dy, dz, dr, dphi, dcosTheta ) are now with respect to the interaction point (as documented). This now makes them different with respect to the cartesian position ( x, y, z ) with respect to the origin.. This is similarly true for ( mcDecayVertexFromIPX, mcDecayVertexFromIPY, mcDecayVertexFromIPZ ) versus ( mcDecayVertexX, mcDecayVertexY and mcDecayVertexZ ).

  • Added the meta variable eclClusterSpecialTrackMatched to check if any track related to an ECL cluster satisfies certain criteria. It can be used to apply an individual track-cluster matching.

  • Added missing error variables DeltaZErr and DeltaBoostErr. Fixed bug in TagV error functions.

  • Replaced pValue (tracking variable) with chiProb (vertexing variable) in the collection of vertexing variables.

  • Replaced pValue (always the track-fit pvalue) with chiProb (context-dependent \(\chi^2\) probability variable) in the vertexing variable collection.

  • Added the meta variables averageValueInList and medianValueInList which calculate the arithmetic mean and median of a given variable in a given particle list.

  • Removed nECLClusters (4407). The number of ECLClusters is not a good measure for anything except the age of the input mdst file. Recent changes to the handling of ECLClusters and their hypotheses have changed this (already arbitrary) number to another (equally arbitrary but different) number.

    Hint

    For most common use cases, the recommended alternative is the total number of photons (unique photon hypothesis clusters) in the event:

    from variables import variables
from stdPhotons import stdPhotons
stdPhotons("cdc", path=mypath) # creates the all and cdc lists
variables.addAlias("nPhotonsAll", "nParticlesInList(gamma:all)")
variables.addAlias("nPhotonsCDC", "nParticlesInList(gamma:cdc)")
variables.addAlias("totalPhotonEECL", "totalEnergyOfParticlesInList(gamma:all)")

  • Deprecated clusterHypothesis following changes to the ECLCluster storage on mdst. Please use clusterHasNPhotons and clusterHasNeutralHadron.

  • Removed ClusterHadronIntensity and clusterUniqueID.

  • Added helix extrapolation from charged tracks.

  • Added clusterMdstIndex variable to get ECLCluster mdst index for particles that do not originate from ECLCluster.

  • Added a convenience M2, that returns the invariant mass squared of a given Particle (determined from the particle’s 4-momentum vector).

  • Added eclClusterOnlyInvariantMass, that returns the invariant mass of the particles, computed with cluster information

  • Added grandDaughterDiffOf, grandDaughterDiffOfPhi, and grandDaughterDiffOfClusterPhi. They behave like daughterDiffOf, but use grandDaughters’ information rather than daughters’.

  • Renamed ClusterHasPulseShapeDiscrimination to clusterHasPulseShapeDiscrimination, PulseShapeDiscriminationMVA to clusterPulseShapeDiscriminationMVA, and ClusterNumberOfHadronDigits to :b2:var`clusterNumberOfHadronDigits`.

  • Removed the misleading variable McFlavorOfTagSide.

  • Added a variable mcFlavorOfOtherB which returns the tag-side flavor by checking the other Upsilon4S daughter without using the ROE. This variable is part of the default flavor_tagging collection.

  • The name of the variable MCTagBFlavor changed to internalTagVMCFlavor. It should be used only by experts for debugging.

  • Added MC flight information variables, mcFlightDistance and mcFlightTime

Modules#

  • Added the new module AllParticleCombiner which is also available via the function modularAnalysis.combineAllParticles. It creates a new Particle as the combination of all unique Particles from the passed input ParticleLists.

  • The ParticleLoader can load Rest Of Event as a particle using a new function modularAnalysis.fillParticleListFromROE. This ROE particle can be combined with any other particle or written down using usual variables.

    Another option is to load missing momentum as a particle by supplementing useMissing = True argument to the function, mentioned above.

  • Fixed a bug in the BestCandidateSelection module: When allowMultiRank=True there always at least two candidates with rank of one, even if they didn’t have the same variable value (4391)

    This affects all users of modularAnalysis.rankByLowest() and modularAnalysis.rankByHighest() if they passed allowMultiRank=True

  • Fixed a bug in the BestCandidateSelection module: now the numBest parameter works as expected.

  • Removal of the unsupported ECLClusterInfoModule.

  • Added vertex.fitPseudo function and a pseudo vertex fitting module to add a covariance matrix when a vertex fit is not possible. E.g. for \(\pi^0\) decays.

  • Added the new module SignalSideVariablesToDaughterExtraInfo. It adds ExtraInfo to a specified particle (typically a daughter on the signal side). The corresponding information is calculated in the RestOfEvent so it is supposed to be executed only in for_each ROE paths.

  • Fixed a bug and extended the functionality of the RestOfEventBuilder module. When providing a ParticleList of composite particles as an additional source for building the ROE, the composite particles are now decomposed and their final state daughters are added to the ROE (unless they are part of the signal side or already present in the ROE). Previously, composite particles were not decomposed and the first composite particle of the first ParticleList of composite particles (and only this one) was always added to the ROE.

  • The module VertexFitUpdateDaughters now always updates the daughters (as advertised).

Modular Analysis#

  • A new boolean argument has been added to the function modularAnalysis.buildRestOfEvent. It is called belle_sources and should be switched to True (default is False) if you are analyzing converted Belle MC or data. It triggers the ROE to be constructed from all tracks and from Belle’s specific gamma:mdst list.

  • Added signal region function modularAnalysis.signalRegion. By default, this function enables a new variable isSignalRegion and excludes the defined area when processing is done on data.

Standard Particle Lists#

  • Updated V0 lists. The standard list became the merged list, a combined list of particles coming from V0 objects merged with a list of particles combined using the analysis ParticleCombiner module.

    stdV0s.stdKshorts returns a Ks list called K_S0:merged stdV0s.stdLambdas returns a Lambda list called Lambda0:merged

    mergedKshorts() and mergedLambdas() are now deprecated, and return the standard lists.

  • The definition of the standard V0 lists slightly changed. For Lambdas, the modularAnalysis.markDuplicate() function is now used to detect duplicates among the same list (V0 and ReconstructDecay), rather than between different lists. For Ks, the modularAnalysis.markDuplicate() function is no longer used.

  • Updated V0 lists by making them explicitly call the appropriate vertex fit.

  • Fixed a bug in the merged Lambda list, which used an incorrect mass window.

  • Fix the stdKlongs.stdKlongs lists.

  • Updated charged standard PID cut values to reflect the correct efficiencies. This recovers the efficiency loss reported in BIIDP-1065.

Tutorials#

  • Fix B2A202 to load Klongs as advertised.

  • Fix B2A801 to use JPsiKs as signal channel and to show how to run on Belle data/MC. Central database global tag updated.

1.7.2. Changes in the framework package#

Job Information File#

basf2 has a new command line parameter --job-information=<filename> to create a json file with some statistics on the processing and detailed information on all created output files.

See also

BELLE2-NOTE-TE-2017-012: Software and Computing Interface

Python Interface#

  • Additional utility function b2test_utils.check_error_free used to automate a code quality (or indeed any) tool check. It requires execution without any output as part of the unit tests. For example the analysis package now requires no doxygen or cppcheck warnings.

  • Additional utility function b2test_utils.configure_logging_for_tests to improve reproducibility of log output for testing purposes by replacing certain setup dependent names of directories with placeholders.

  • The Pager class has been moved to a new module terminal_utils which now also contains a new class InputEditor to open a editor when asking the user for a long message similar to git commit

  • The compatibility imports of sys, os and the fw object have been removed from basf2. Every method which was previously part of the fw object is now just a function in the basf2 module.

Core Framework#

  • Addition of helpful StoreArray::hasRelationTo and StoreArray::hasRelationFrom methods.

  • Deprecated static methods to register StoreArray instances have been removed.

Command line Tools#

  • b2file-metadata-add will now only update the file in the FileCatalog if it is already registered. It will also now correctly remove the old LFN from the file catalog.

  • b2file-merge will by default no longer register files in the file catalog. You can either supply --add-to-catalog as argument or run b2file-catalog-add on the output file. Use the latter if you want to write scripts which work also with older releases.

  • b2file-normalize has been added to remove timestamps and similar non-reproducible features from ROOT files.

  • b2file-remove-branches has been added to remove obsolete branches from old files.

  • the commands of the b2conditionsdb tool are now also available with dashed version, e.g. b2conditionsb-tag-show

  • A number of old tool names which were kept for compatibility have been removed. The old tool names still exist but raise an error pointing to the new name.

Modules#

  • b2:mod:RootOutput module: changed default value for updateFileCatalog to False. This will avoid creating Belle2FileCatalog.xml or warning about overwriting entries in the file catalog by default. The file catalog is only needed when loading parent files and can always be created later using b2file-catalog-add

    See also

    b2file-catalog-add

1.8. Changes since release-02-01#

Moved to C++17

The whole software including the ROOT in the externals is now compiled using the C++17 standard. This should not affect many users but there are a few rare cases where this might lead to compilation problems of analysis code as some deprecated features have been removed. The most notable are

  • throw(TypeName) exception specifiers, just remove them.

  • std::auto_ptr which should be replaced by std::unique_ptr

  • some older parts of the <functional> header.

In particular if you compile a standalone program that links against the ROOT in the Belle2 externals this now also needs to be compiled in C++17 mode. You can do this by adding -std=c++17 to the compiler arguments.

Note

It’s best to directly pass the output of root-config --cflags to the compiler. That way you always pass the correct flags needed for the particular ROOT version setup.

Build system moved to Python3

This is a major update of SCons but most users should not notice any difference except for two instances:

  • If you update an existing working directory from an older release you might run into an error

    scons: *** [...] TypeError : intern() argument 1 must be string, not unicode
scons: building terminated because of errors.
TypeError: intern() argument 1 must be string, not unicode:
[...]

    In this case please remove all .scon* files in the top level of your software directory and rerun scons

  • In the unlikely case that you have custom SConscript files which are not Python 3 compatible you will have to update them.

1.8.1. Changes in the analysis package#

Variables#

There have been improvements and additions to the variables available in the variable manager. Some older, unhelpful, or deprecated variables have been removed, but this should be less than previous major releases. As usual, please ask at https://questions.belle2.org if anything is unclear or your favourite variable seems to have been removed. There is likely a good reason.

New Modular Analysis Tools#

Fitters#

  • Many improvements to TreeFitter, which is now the recommended vertex fitter almost all use cases, even for simple vertices with two tracks. Please refer to the TreeFitter documentation for full details.

    See also

    This TreeFitter tutorial (October 2018).

  • Bug fixes to OrcaKinFit.

  • KFit is now accessible from other basf2 modules.

Loading V0 Particles#

  • When loading V0 particles (i.e. \(K_S^0\), \(\Lambda^0\), or converted photons) using the ParticleLoader (fillParticleList) you must now specify the daughters in a decay string. For example, to load \(\Lambda^0\to p^+\pi^-\) decays from V0s:

    from modularAnalysis import fillParticleList
fillParticleList('Lambda0 -> p+ pi-', '0.9 < M < 1.3', path=mypath)

Tutorials and Examples#

  • The style of many of the tutorial scripts have been updated to assist with and provide examples of these new changes. See:

    $BELLE2_RELEASE_DIR/analysis/examples/tutorials

1.8.2. Changes in the framework package#

Python Interface#

The basf2 python interface has been restructured into multiple files to ease maintenance. For the user this should be transparent. Some changes the user might notice are

  • There is now a new utility function basf2.find_file to allow looking for files in the release directory or separate examples or validation directories.

  • There is now an automatic Jupyter integration: Calling basf2.process in a Jupyter notebook should now automatically run the processing in a separate process and show a nice progress bar.

  • The obsolete “fw” object has been deprecated and all functionality which previously was accessed using basf2.fw.* is now directly accessible as basf2.*. If you use basf2.fw you will get a deprecation warning.

  • When using from basf2 import * the sys and os packages were also silently imported and available in the current script. This has been deprecated. In general, using import * is not recommended but if you have to use it and use the sys or os module in the script please make sure you import them yourself after from basf2 import *

  • Display and colouring of log messages in Jupyter has been significantly improved and should now be much faster.

  • There’s a new implementation for pager.Pager which can also show output from C++ programs and will display the output incrementally instead of waiting for all output before showing everything.

Command Line Tools#

  • b2file-check now supports files with zero events correctly.

  • b2file-merge now checks that real/MC data flag is consistent for all input files and refuse to merge mixed real/MC files.

  • The subcommands iov and diff of b2conditionsdb have improved output and learned the new option --human-readable to convert the IntervalOfValidity numbers to easier to read strings.

  • There is a new sub command dump for b2conditionsdb to dump the contents of a payload file in a human readable form on the terminal for quick inspection.

  • There is a new command b2conditionsdb-extract which allows to convert a payload to a TTree with one entry per requested run number. This allows to easily monitor how payloads change over the course of time.

  • There is a new command b2conditionsdb-recommend which will recommend users a global tag to use when processing a given input file.

  • There is a new command b2conditionsdb-request to allow requesting the inclusion of locally prepared database payload into the official global tags.

Core Framework#

  • Environment::isMC() is now available to consistently distinguish between real and MC data. To use it in C++ please use

    #include <framework/core/Environment.h>
bool isMC = Environment::Instance().isMC();

    and for use in python

    from ROOT import Belle2
isMC = Belle2.Environment.Instance().isMC();

  • We have now a new prototype for advanced multi processing using ZMQ. It is disabled by default but can be activated using

    from ROOT import Belle2
env = Belle2.Environment.Instance().setUseZMQ(True)

    See also

    Pull request [PR#2790]

  • There is now support for named relations to allow multiple relations between the same pair of StoreArray.

Core Modules#

  • The ProgressBar module will now notice if the output is written to a log file instead of the terminal and behave accordingly which should clean up logfiles considerably when this module is used.

  • The RootOutput module now allows to split output files after a certain file size is reached using the outputSplitSize parameter.

    Warning

    This will set the amount of generated events stored in the file metadata to zero as it is not possible to determine which fraction ends up in which output file.

    Also the user can now choose the compression algorithm to choose either no compression, zlib, LZMA or LZ4 compression. LZ4 is a newer compression standard with slightly worse compression than zlib but much faster decompression speed.

  • removed CrashHandler module

  • removed FileLogger module

Conditions Database:

  • Database objects are now immutable (const) to prevent accidental modification of conditions data.

  • The old and outdated fallback database in framework/data/database.txt has been removed. If you still set this by hand in your steering file your script will fail. Please use /cvmfs/belle.cern.ch/conditions if you really have to set a fallback database manually.

  • The RunInfo database object which is supposed to contain all necessary information about each run now has support to contain the trigger pre-scale information.

Logging System#

  • We now have “Log message variables” which allow to send the same log message with varying content. This greatly helps with filtering log messages as it allows to group messages which have the same content and just differ in their variables. In C++ they can be used by adding a LogVar instance to the output,

    B2INFO("This is a log message" << LogVar("number", 3.14) << LogVar("text", "some text"));

    while in python the variables can be given as additional keyword arguments,

    basf2.B2INFO("This is a log message", number=3.14, text="some text")

    In both cases the names of the variables can be chosen feely and the output should be something like

    [INFO] This is a log message
        number = 3.14
        text = some text

    See also

    Log Variables

  • The logging system is now able to send its message to python sys.stdout objects to allow intercepting log messages in python. To enable please set basf2.logging.enable_python_logging to True. This is automatically enabled when running inside of a jupyter notebook.

  • Log messages can also be formatted as json objects where each log message will be printed as a one line json object to allow parsing of logfiles using scripts.

    See also

    basf2.logging.add_json()

Utilities#

  • We have a new and advanced formula parser implementation in the framework package. It manages to handle ** correctly as in python and now allows using normal parenthesis for grouping operations in addition to square brackets.

    See also

    formula variable

  • We now provide a variety of RAII scope guards to free or restore a resource or value when the guard object goes out of scope. For example to make sure a variable is reset to is original value one could use

    #include <framework/utilities/ScopeGuard.h>

int main() {
  int myValue{5};
  {
    auto guard = Belle2::ScopeGuard::guardValue(myValue, 32);
    // now myValue is 32
  }
  // now myValue is reverted to 5 independently of how the scope
  // is left (normal, return statement, exception)
}

    We provide convenience functions to guard simple values, pointer deletion, output stream flags and the current working directory but the interface is general enough that almost anything can be guarded by this ScopeGuard object.

  • There is now RootFileManager to allow multiple modules to write to the same root output file. It will take care to open the file when the first module requests it and close it when the last module is finished with the root file. The primary use case is for the VariablesToNtuple and similar modules to allow having multiple ntuples or trees in the same root file.

  • We now have a implementation of the c++17 std::visit overloaded pattern in framework/utils/Utils.h called Belle2::Utils::VisitOverload.