Metadata-Version: 2.4
Name: galechurch
Version: 0.1.0
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Text Processing :: Linguistic
Requires-Dist: numpy>=1.23 ; extra == 'calibrate'
Requires-Dist: numpy>=1.23 ; extra == 'dev'
Requires-Dist: pytest>=7.0 ; extra == 'dev'
Provides-Extra: calibrate
Provides-Extra: dev
License-File: LICENSE
Summary: Fast, multi-core Gale-Church sentence alignment
Author: Steinthor Steingrimsson
License-Expression: Apache-2.0
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://github.com/steinst/galechurch
Project-URL: Repository, https://github.com/steinst/galechurch

# Gale-Church Sentence Alignment

A fast, multi-core implementation of the Gale-Church algorithm for aligning sentences in bilingual
corpora, with a Rust core (via [PyO3](https://pyo3.rs)/[maturin](https://www.maturin.rs)) exposed as
the `galechurch` pip package.

## Installation

```bash
pip install galechurch
# or, for the parameter-calibration tools (needs numpy):
pip install galechurch[calibrate]
```

Prebuilt wheels are published for common platforms (Linux x86_64/aarch64, macOS x86_64/arm64,
Windows x86_64); an sdist is also published, so `pip install` falls back to a source build
(requires a Rust toolchain) on any platform without a matching wheel.

## Library usage

```python
import galechurch

result = galechurch.align(source_sentences, target_sentences)
print(result.to_pharaoh_lines())  # ["[0]:[0]", "[1]:[1, 2]", ...]
```

`bead_costs`, `mean_xy`, and `variance_xy` default to the values the original `align` program shipped
with (estimated from the UBS trilingual corpus; Gale & Church 1993, Table 5 and section 4), and can be
overridden per call:

```python
result = galechurch.align(
    source_sentences,
    target_sentences,
    bead_costs={(1, 1): 0.0, (2, 2): 400.0},  # only override the keys you need; rest keep defaults
    mean_xy=1.05,
    variance_xy=7.2,
)
```

For a whole corpus of matched source/target file pairs, `galechurch.align_corpus` aligns every block
of every file in a single call, using all available CPU cores:

```python
from galechurch import align_corpus

results = align_corpus("./eval_data/parice/eng", "./eval_data/parice/isl", "./output/parice")
```

## CLI usage

The `galechurch` console script mirrors the library API and the original CLI's flag names:

```bash
galechurch -src ./eval_data/parice/eng -trg ./eval_data/parice/isl -out ./output/parice
```

## Calibrating for a new language pair or data type

Given a hand-aligned gold corpus, `galechurch-calibrate` (or `galechurch.calibrate` as a library)
estimates bead costs and length-cost parameters following the procedure in the original paper
(section 4, Table 5), and saves them as a reusable JSON profile:

```bash
galechurch-calibrate -src mydata/source.txt -trg mydata/target.txt -gold mydata/gold.txt -out my_profile.json
galechurch --profile my_profile.json -src ./mydata/source_dir -trg ./mydata/target_dir -out ./output
```

## Evaluating alignments

`evaluate.py` (a dev-only script, not part of the pip package) compares generated alignments against
gold-standard files, reporting strict and lax precision/recall/F1:

```bash
python3 evaluate.py -t ./output/parice -g ./eval_data/parice/gold
```

You can also evaluate individual files:
```bash
python3 evaluate.py -t ./output/parice/u_1.txt -g ./eval_data/parice/gold/u_1.txt
```

The `./eval_data` folder contains two evaluation sets used by the test suite: the German-French
evaluation set from the Text+Berg corpus, and an English-Icelandic evaluation set from the ParIce
corpus.

## Citations

If you use the code in this repository, please cite the original Gale-Church paper:
```bibtex
@article{gale-church-1993-program,
    title = "A Program for Aligning Sentences in Bilingual Corpora",
    author = "Gale, William A.  and Church, Kenneth W.",
    journal = "Computational Linguistics",
    volume = "19",
    number = "1",
    year = "1993",
    url = "https://aclanthology.org/J93-1004",
    pages = "75--102",
}
```

### Evaluation sets
If you use the evaluation sets provided in this repository, please consider citing the original papers where the evaluation sets were first used. For the German-French evaluation set, please cite the Bleualign paper:

#### BleuAlign

```bibtex
@inproceedings{Sennrich2010MTbasedSA,  
    title = {{MT-based Sentence Alignment for OCR-generated Parallel Texts}},
    author = "Rico Sennrich and Martin Volk",  
    booktitle = "Proceedings of the 9th Conference of the Association for Machine Translation in the Americas: Research Papers",
    address = "Denver, Colorado",
    year = "2010",
    publisher = "Association for Machine Translation in the Americas",
    url = "https://aclanthology.org/2010.amta-papers.14",
}
```

#### ParIce evaluation set
The parice folder contains an evaluation set for English-Icelandic sentence alignment from 10 aligned documents in five subcorpora of the ParIce corpus. The evaluation set is [distributed](https://repository.clarin.is/repository/xmlui/handle/20.500.12537/150) under a CC BY 4.0 license. If you use that evaluation set, please consider citing the ParIce paper and the SentAlign paper where the evaluation set was first used.

The ParIce paper:

```bibtex
@inproceedings{barkarson-steingrimsson-2019-compiling,
    title = {{Compiling and Filtering {P}ar{I}ce: An {E}nglish-{I}celandic Parallel Corpus}},
    author = "Barkarson, Starka{\dh}ur and Steingr{\'\i}msson, Stein{\th}{\'o}r",
    booktitle = "Proceedings of the 22nd Nordic Conference on Computational Linguistics",
    month = sep # "{--}" # oct,
    year = "2019",
    address = "Turku, Finland",
    publisher = {Link{\"o}ping University Electronic Press},
    url = "https://aclanthology.org/W19-6115",
    pages = "140--145",
}
```

