Ep 54 — Open Source Tools (Part 2): HyperSpy, Orange-Spectroscopy

Series: Infrared Spectroscopy Encyclopedia: From Principles to Practice
Chapter: Chapter 5 · Instruments and Tools — Open Source Ecosystem (Ep 46–55 later part)
Target Audience: Technicians wanting to improve data processing efficiency, micro-IR / FPA imaging data users, chemometrics enthusiasts, analysts with zero programming needs
Prerequisites: Ep 18 (Spectrum Processing), Ep 19 (Library Search), Ep 20 (Quantitative Analysis), Ep 53 (Open Source Tools Part 1)
Reading Time: Approximately 45 minutes

Prologue: A "4 GB" FPA Imaging Dataset

Xiao Wang from a materials lab used a Bruker Hyperion 3000 with a 64×64 FPA detector to scan a micro-IR image of a polymer blend film. After exporting the data, the file size was 4 GB—4096 spectra, each from 4000–900 cm⁻¹ with 1769 data points. The OPUS software could open it, but each operation required a 30-second wait. The analysis she wanted to perform was complex:

1. Plot the chemical image at 1740 cm⁻¹ (polyester C=O);
2. Decompose into 3 pure component spectra using PCA;
3. Perform curve fitting on each pixel's spectrum to quantify the polyester/polyamide ratio;
4. Visualize the results overlaid on the sample's optical photo.

The built-in functions of OPUS could barely complete step 1. Her supervisor recommended HyperSpy—the open-source benchmark for multidimensional spectral data analysis. One week later, Xiao Wang completed all 4 steps of analysis using HyperSpy, turned the code into a reusable workflow, and subsequently processed each imaging dataset within 5 minutes [1].

"For multidimensional spectral data (imaging, time series, depth scans), HyperSpy is a very useful open-source tool. It treats the 'data cube' as a first-class citizen."
— Adapted from Microscopy & Microanalysis 2022 review [1]

This episode will systematically review four open-source spectral tools: HyperSpy (benchmark for multidimensional spectral imaging analysis), Orange-Spectroscopy (drag-and-drop visual workflow), SpectraKit (lightweight core operations), NIRS4ALL (deep learning + PLS unified framework). Each comes with a GitHub link, star count, and hands-on code.

💡 Positioning of this episode: Ep 53 covered general frameworks, Ep 54 turns to "specialized" tools: HyperSpy specializes in multidimensional imaging, Orange specializes in zero-programming workflows. They complement each other, forming a complete open-source spectral tool ecosystem.

1. HyperSpy: Benchmark for Multidimensional Spectral Imaging Analysis

1.1 Project Overview

HyperSpy is an open-source Python library focused on multidimensional spectral data analysis, widely used in electron microscopy spectroscopy and multidimensional spectral analysis; when used for FTIR FPA, data format adaptation and IR-specific preprocessing are required [1][2][3].

• GitHub: https://github.com/hyperspy/h…
• Documentation: https://hyperspy.org/
• Stars: Approximately 560+ (please refer to the GitHub page for current count; "highest among spectral tools" should not be absolute)
• License: GPL-3.0
• First Release: 2011
• Python Version: 3.9+
• Supporting Institutions: Diamond Light Source (UK), ICMCB (France), etc.

Figure 1: HyperSpy GitHub homepage (Source: https://github.com/hyperspy/h…)

1.2 Core Data Structure: Signal

The core of HyperSpy is the Signal class—an N-dimensional data container that distinguishes between "navigation axes" and "signal axes" [2][3]:

import hyperspy.api as hs

# Load FPA imaging data (64×64 pixels × 1769 wavenumbers)
s = hs.load('polymer_blend.hyspermap')

# Check data dimensions
print(s)
# <Signal2D, title: , dimensions: (64, 64|1769)>
#                    ↑navigation axes↑      ↑signal axis↑

Meaning of (64, 64|1769) [2]:

• Navigation axes: 64×64 spatial pixels;
• Signal axis: 1769 wavenumber points per pixel.

This "navigation vs signal" distinction is HyperSpy's core innovation, making multidimensional data manipulation intuitive [2][3]:

# Extract a single pixel spectrum
s.inav[10, 20].plot()  # Spectrum at pixel (10, 20)

# Extract a chemical image at a specific wavenumber
s.isig[1740.0].plot()  # Chemical image at 1740 cm⁻¹ (T demeanor)

# Extract a rectangular region
s.inav[10:20, 30:40].plot()  # Average spectrum of that rectangular region

1.3 Full Workflow Coverage

HyperSpy covers the complete workflow of multidimensional spectral analysis [2][3]:

Load → Preprocess → Decompose (PCA/ICA) → Fit → Visualize → Export
 │       │            │                  │      │          │
 │       │            │                  │      │          └ HDF5, npy, TIFF
 │       │            │                  │      └ plot, plot_images, plot_spectra
 │       │            │                  └ Multi-peak fitting, model fitting
 │       │            └ PCA, ICA, VCA, NMF, BSS
 │       └ Baseline correction (pybaselines integration), smoothing, normalization
 └ Multiple formats (HSPY, HDF5, Bruker, Thermo, EDAX, etc.)

1.4 Hands-on: FPA Imaging Data Cube Processing

The following complete example demonstrates the workflow for processing FPA imaging data with HyperSpy [1][2][3]:

import hyperspy.api as hs
import numpy as np

# ========== 1. Load Data ==========
s = hs.load('polymer_blend.hyspermap')
print(s)  # (64, 64|1769)

# Set signal axis to wavenumber
s.axes_manager.signal_axes[0].offset = 900
s.axes_manager.signal_axes[0].scale = 0.25  # 0.25 cm⁻¹/point
s.axes_manager.signal_axes[0].units = 'cm⁻¹'
s.axes_manager.signal_axes[0].name = 'Wavenumber'

# Set navigation axes
s.axes_manager.navigation_axes[0].units = 'μm'
s.axes_manager.navigation_axes[0].scale = 5.5  # 5.5 μm/pixel
s.axes_manager.navigation_axes[0].name = 'X'
s.axes_manager.navigation_axes[1].units = 'μm'
s.axes_manager.navigation_axes[1].scale = 5.5
s.axes_manager.navigation_axes[1].name = 'Y'

# ========== 2. Preprocessing ==========

# 2.1 Crop region of interest (1800-900 cm⁻¹)
s_cut = s.isig[1800.0:900.0]

# 2.2 Baseline correction (using pybaselines)

from pybaselines.whittaker import arpls

def baseline_pixel(y):
    return arpls(y, lam=1e7)[0]

# Baseline correction for each pixel
s_bc = s_cut.map(baseline_pixel, inplace=False, ragged=False)

# 2.3 Normalization (independent per pixel)
s_norm = s_bc / s_bc.max(axis=-1)  # Normalize along signal axis

# ========== 3. PCA Decomposition ==========

# 3.1 PCA
s_norm.decomposition()
s_norm.plot_explained_variance_ratio()

# 3.2 Take first 3 principal components
sc = s_norm.get_decomposition_model(3)
sc.plot()

# 3.3 ICA Deconvolution (BSS)
s_bss = s_norm.blind_source_separation(3)
s_bss.plot()

# ========== 4. Chemical Imaging ==========

# 4.1 Chemical imaging at 1740 cm⁻¹ (Polyester C=O)
s_norm.isig[1740.0].plot(cmap='viridis', colorbar=True,
                         title='Polyester distribution (1740 cm⁻¹)')

# 4.2 Chemical imaging at 1630 cm⁻¹ (Polyamide amide I)
s_norm.isig[1630.0].plot(cmap='plasma', colorbar=True,
                         title='Polyamide distribution (1630 cm⁻¹)')

# ========== 5. Multi-peak Fitting (Quantification per pixel) ==========

# Create model
m = s_norm.create_model()

# Add 3 Gaussian peaks
m.append(hs.model.components1D.GaussianHFWHM(center=1740, fwhm=20))
m.append(hs.model.components1D.GaussianHFWHM(center=1630, fwhm=25))
m.append(hs.model.components1D.GaussianHFWHM(center=1540, fwhm=25))

# Fit all pixels
m.multifit()

# Extract intensity maps for each component
polyester_map = m.components.GaussianHFWHM.A.map
polyamide1_map = m.components.GaussianHFWHM_0.A.map
polyamide2_map = m.components.GaussianHFWHM_1.A.map

# Calculate polyester/polyamide ratio
ratio_map = polyester_map / (polyamide1_map + polyamide2_map)
hs.signals.Signal2D(ratio_map).plot(cmap='RdYlBu',
    title='Polyester / Polyamide ratio')

1.5 HyperSpyUI: Graphical Interface

HyperSpy also provides a graphical interface HyperSpyUI, enabling users who are not familiar with Python to use HyperSpy's core features [2]:

• Load and visualize multidimensional data;
• Basic preprocessing (smoothing, derivatives, baseline);
• PCA / ICA decomposition;
• Multi-peak fitting;
• Export images.

Figure 2: HyperSpyUI interface (Source: https://github.com/hyperspy/h…)

When to use HyperSpyUI [2]:

• Quick analysis without writing code;
• Exploratory data analysis;
• Teaching (let students start with GUI then code).

1.6 When to Choose HyperSpy

Suitable for [1][2][3]:

• Micro-FTIR imaging (FPA, scanning type);
• Multidimensional data such as time series, depth profiles;
• PCA / ICA / NMF decomposition;
• Multi-peak fitting (independent fitting per pixel);
• Integration with electron microscopy data (same project family).

Not suitable for:

• Routine processing of single spectra (use SpectroChemPy for lighter workload);
• Drag-and-drop workflow (use Orange);
• Large-scale chemometric modeling (use scikit-learn directly).

2. Orange-Spectroscopy: Drag-and-Drop Visual Workflow

2.1 Project Overview

Orange-Spectroscopy is a spectral add-on for the Orange3 data mining framework, developed by the Biolab laboratory at the University of Ljubljana, Slovenia, focusing on zero-coding, drag-and-drop workflows [4][5].

• GitHub: https://github.com/Quasars/or…
• Documentation: https://orange-spectroscopy.r…
• Stars: ~140 (as of 2024)
• License: GPL-3.0
• First release: 2016
• Python version: 3.8+

Figure 3: Orange-Spectroscopy GitHub page (Source: https://github.com/Quasars/or…)

2.2 Core Feature: Visual Workflow

Orange's core philosophy is "connecting icons instead of writing code" [4][5]. A complete spectral analysis workflow in Orange looks like this:

[File]      → [Spectra]   → [Preprocess] → [PCA]    → [Scatter Plot]
   │                            │             │              │
   └→ [Datasets]                └→ [Cut]      └→ [PLS]        └→ [Predictions]
                                └→ [Baseline]
                                └→ [Normalize]

Each box is a widget, and connections represent data flow. Users drag widgets, connect them, and adjust parameters without writing any code [4].

Figure 4: Orange-Spectroscopy workflow interface (Source: https://orange-spectroscopy.r…)

2.3 Main Widgets

Orange-Spectroscopy provides widgets in four categories [4][5]:

① Data I/O

• File: read CSV, JCAMP-DX, HDF5, SPC, etc. formats;
• Datasets: built-in NIR, IR example datasets;
• Save Data: save as CSV, HDF5.

② Preprocessing

• Cut: select wavenumber range;
• Baseline: baseline correction (polynomial, Rubber Band, ALS);
• Normalize: normalization (vector, area, maximum);
• Smooth: SG smoothing;
• Derivative: derivative;
• Scatter Correction: SNV, MSC.

③ Analysis

• PCA: Principal Component Analysis;
• PLS: Partial Least Squares regression;
• K-Means: clustering;
• Hierarchical Clustering: hierarchical clustering;
• Random Forest: random forest classification.

④ Visualization

• Spectra: spectral overlay;
• Scatter Plot: scatter plot;

• Heat Map: heat map;

• Line Plot: line chart.

2.4 Hands-on: From Import to PLS Model

Below, we build a complete NIR modeling workflow using Orange-Spectroscopy (GUI steps) [4][5]:

Step 1 — Load Data

1. Drag the File widget onto the canvas;
2. Double-click, select nir_mixture.csv (200 samples × 700 wavelengths + 1 target value 'protein');
3. Connect the output to the Spectra widget to view the spectra.

Step 2 — Preprocessing

1. Drag the Preprocess Spectra widget;
2. Chain operations: Cut (1100-2500 nm) → Scatter Correction (SNV) → Derivative (SG, order=2, window=15);
3. Connect output to Spectra widget to compare before and after processing.

Step 3 — PCA Exploration

1. Drag the PCA widget;
2. Set Components = 10;
3. Connect output to Scatter Plot, plot PC1 vs PC2 scatter plot;
4. Color by target value 'protein' to see if there is a trend.

Step 4 — PLS Modeling

1. Drag the PLS widget;
2. Set Components = 10;
3. Input: preprocessed spectra + target value 'protein';
4. Connect output to Predictions widget to view predicted vs. actual;
5. Use the Test & Score widget for 10-fold cross-validation, output R², RMSE.

Step 5 — Model Application

1. Drag a second File widget to load new samples;
2. Process through the same Preprocess widget (keep parameters consistent);
3. Connect to the PLS widget (as input);
4. The Predictions widget outputs the prediction results.

The entire workflow can be built in 30 minutes, completely code-free [4][5].

2.5 When to Choose Orange-Spectroscopy

Suitable for [4][5]:

• Zero-code users;
• Classroom teaching (students get started in 30 minutes);
• Rapid prototyping analysis;
• Exploratory data analysis;
• Interdisciplinary collaboration (chemistry + computer science + business);
• Routine chemometrics such as PLS, PCA.

Not suitable for:

• Complex custom algorithms (use HyperSpy / SpectroChemPy);
• Multidimensional imaging data (use HyperSpy);
• Deep learning (use NIRS4ALL);
• GMP production environments (no compliance certification).

3. SpectraKit (pyspectrakit): Lightweight Core Operations

3.1 Project Overview

SpectraKit (also known as pyspectrakit) is a lightweight spectral analysis package developed by ktubhyam, with core dependencies only on NumPy + SciPy, focusing on 'teaching atomic operations' [2][6].

• GitHub: https://github.com/ktubhyam/s…
• Stars: ~50 (as of 2024)
• License: MIT
• First release: 2021
• Python version: 3.7+

Figure 5: SpectraKit GitHub homepage (Source: https://github.com/ktubhyam/s…)

3.2 Key Features

① Minimal Dependencies [6]

# Only need NumPy + SciPy
import numpy as np
from scipy import signal

② Clear Atomic Operations [6]
Each function corresponds to a clear atomic spectral operation:

import pyspectrakit as psk

# 1. Load
wavenumber, absorbance = psk.load('sample.csv')

# 2. Baseline correction (linear)
baseline = psk.baseline_linear(absorbance, wavenumber)
corrected = absorbance - baseline

# 3. SG smoothing
smoothed = psk.smooth_sg(corrected, window=9, order=2)

# 4. First derivative
d1 = psk.derivative(smoothed, order=1, window=11, polyorder=3)

# 5. Peak detection
peaks, properties = psk.find_peaks(smoothed, height=0.05, prominence=0.02)
print(f"Peaks at: {wavenumber[peaks]} cm⁻¹")

# 6. Peak area
area = psk.peak_area(smoothed, wavenumber, peak=1740, window=20)
print(f"Peak area at 1740 cm⁻¹: {area}")

③ Suitable for Teaching Atomic Operations [6]

• Each function implementation is very short (10–30 lines);
• Suitable for teaching: let students read source code and understand principles;
• It is a transition from 'using tools' to 'writing tools'.

3.3 When to Choose SpectraKit

Suitable for [6]:

• Teaching (let students understand the principle of each operation);
• Lightweight tasks (no need for complex frameworks);
• Embedding into other systems (few dependencies);
• Learning how to write your own spectral processing functions.

Not suitable for:

• Large-scale analysis in production;
• Reading multi-vendor formats;
• Advanced chemometrics.

4. NIRS4ALL: Unified Framework for Traditional PLS and Deep Learning

4.1 Project Overview

NIRS4ALL developed by GBeurier, aims to unify 30+ spectral transformation methods, traditional PLS, and deep learning into one framework [2][7].

• GitHub: https://github.com/GBeurier/n…
• Stars: ~80 (as of 2024)
• License: MIT
• First release: 2019
• Python version: 3.7+

Figure 6: NIRS4ALL GitHub homepage (Source: https://github.com/GBeurier/n…)

4.2 Key Features

① 30+ Spectral Transformations [7]
Built-in spectral transformation methods in NIRS4ALL:

• Basic: log, inverse, square root;
• Normalization: SNV, MSC, area normalization, maximum normalization;
• Derivative: first, second, SG derivative;
• Baseline: polynomial, ALS;
• Scatter correction: SNV, MSC, Detrend;
• Wavelet: continuous wavelet transform;
• Feature selection: CARS, VIP, SPA.

② Unified API for Traditional PLS and Deep Learning [7]

from nirs4all import Pipeline
from nirs4all.models import PLSR, CNN, LSTM, Transformer
from nirs4all.transforms import SNV, SG_Derivative, ALS

# Define pipeline (traditional PLS)
pipeline_pls = Pipeline([
    ('snv', SNV()),
    ('derivative', SG_Derivative(window=15, order=2)),
    ('pls', PLSR(n_components=10))
])

# Same API, switch to CNN
pipeline_cnn = Pipeline([
    ('snv', SNV()),

('derivative', SG_Derivative(window=15, order=2)),
('cnn', CNN(n_filters=16, kernel_size=5, epochs=100))
])

Training

pipeline_pls.fit(X_train, y_train)
pipeline_cnn.fit(X_train, y_train)

Prediction

y_pred_pls = pipeline_pls.predict(X_test)
y_pred_cnn = pipeline_cnn.predict(X_test)

**③ Deep learning is suitable for large datasets** [7]
- When the number of samples > 10,000, deep learning begins to surpass PLS;
- NIRS4ALL provides an end-to-end workflow: preprocessing → model → evaluation;
- Built-in PyTorch backend.

### 4.3 When to choose NIRS4ALL

**Suitable for** [7]:
- Large datasets (thousands to tens of thousands of samples);
- Deep learning modeling;
- Systematic comparison of traditional methods vs. deep learning;
- Wanting to try multiple models with a unified API.

**Not suitable for**:
- Small datasets (deep learning easily overfits);
- Simple analysis tasks (use Orange or spectrapepper);
- Multidimensional imaging data.

---

## V. Comparison of Four Tools

### 5.1 Feature Matrix

| Dimension | HyperSpy | Orange-Spectroscopy | SpectraKit | NIRS4ALL |
|------|----------|---------------------|------------|----------|
| **Stars** | ~560 | ~140 | ~50 | ~80 |
| **Positioning** | Multidimensional imaging | Drag-and-drop workflow | Teaching/lightweight | Deep learning |
| **Programming required** | Python | Zero-code | Python | Python |
| **Multidimensional data** | ⭐⭐⭐⭐⭐ | ⭐⭐ | ⭐ | ⭐⭐ |
| **Preprocessing** | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ (30+) |
| **Machine learning** | PCA/ICA | PLS/PCA/RF | None | PLS/CNN/LSTM |
| **GUI** | HyperSpyUI | Main GUI | None | None |
| **Teaching value** | Medium (multidimensional) | High (zero-code) | Very high (atomic operations) | Medium |
| **Production environment** | Suitable | Suitable (rapid prototyping) | Suitable (embedded) | Suitable (big data) |

> Table 1: Comparison of four open-source spectroscopy tools

### 5.2 Selection Decision Tree

What are your needs?
│
├─ Micro-infrared / FPA imaging data
│ └─ HyperSpy
│
├─ Zero programming / teaching / rapid prototyping
│ └─ Orange-Spectroscopy
│
├─ Teaching / understanding atomic operations
│ └─ SpectraKit
│
├─ Large dataset / deep learning
│ └─ NIRS4ALL
│
├─ Full-workflow chemical spectroscopy (combined with previous episode)
│ └─ SpectroChemPy (Ep 53)
│
├─ Baseline correction only (combined with previous episode)
│ └─ pybaselines (Ep 53)
│
└─ Cross-platform / online viewing
└─ ftir.fun

> Figure 7: Decision tree for selecting open-source spectroscopy tools (combining Ep 53 + Ep 54)

### 5.3 Combined Use Case

**Real workflow**: Food adulteration detection, from FPA imaging to PLS model [1][4][5]:

```python
# ========== Stage 1: HyperSpy processing FPA imaging ==========
import hyperspy.api as hs

# Load imaging data
s = hs.load('food_sample.hyspermap')

# Preprocessing + PCA
s.isig[1800.0:900.0]
s.map(lambda y: y - arpls(y, lam=1e7)[0])  # baseline
s.decomposition()
s_bss = s.blind_source_separation(3)  # 3 pure components

# Export 3 component spectra as CSV
for i, comp in enumerate(s_bss):
    comp.save(f'component_{i}.csv')

# ========== Stage 2: Orange modeling ==========
# Load the CSV exported from HyperSpy into Orange (GUI operation)
# - File widget to load pure component spectra
# - Preprocess: SNV + 2nd derivative
# - PLS: build quantitative model of pure components vs. adulteration ratio
# - Test & Score: 10-fold cross-validation

# ========== Stage 3: NIRS4ALL large data modeling ==========
# When number of samples > 10000
from nirs4all import Pipeline
from nirs4all.models import CNN
from nirs4all.transforms import SNV, SG_Derivative

pipeline = Pipeline([
    ('snv', SNV()),
    ('derivative', SG_Derivative(window=15, order=2)),
    ('cnn', CNN(n_filters=32, epochs=200))
])
pipeline.fit(X_train_large, y_train_large)

This combination of "HyperSpy for component extraction + Orange for PLS + NIRS4ALL for deep learning" is a common practice in modern spectral data analysis [1][5][7].

VI. "Pitfalls" of Open-Source Tools

6.1 Pitfall 1: HyperSpy Learning Curve

Scenario: Users accustomed to OMNIC's "acquire and view" are confused by concepts like Signal / navigation axis when first using HyperSpy [2].

Countermeasures:

• First look at the official tutorial (hyperspy.org);
• Use HyperSpyUI to become familiar with concepts;
• Start with single spectrum (1D Signal), then move to imaging (2D Signal + 1D navigation);
• Do not jump into processing 64×64 FPA data.

6.2 Pitfall 2: Orange Version Compatibility

Scenario: After upgrading Orange3, old workflows cannot be opened [4].

Countermeasures:

• Orange workflows (.ows files) have poor cross-version compatibility;
• Back up workflows before upgrading;
• Keep the corresponding Orange version for critical workflows.

6.3 Pitfall 3: Memory Explosion

Scenario: Loading 64×64 FPA data cube uses 8 GB memory, crashes during PCA [2].

Countermeasures:

# Use out-of-core computation
s.decomposition(algorithm='svd', output_dimension=10)

# Or first reduce data with ROI
s_roi = s.inav[10:30, 10:30]  # 20x20 pixels
s_roi.decomposition()

6.4 Pitfall 4: Model Overfitting (NIRS4ALL)

Scenario: CNN achieves R²=0.99 on training set, R²=0.3 on test set [7].

Countermeasures:

• Check sample size (< 1000, deep learning not recommended);
• Strengthen regularization (Dropout, Weight Decay);
• Use K-fold cross-validation instead of single train/test split;
• Compare with PLS baseline; deep learning must be significantly better to be meaningful.

6.5 Pitfall 5: Data Interoperability between Orange and HyperSpy

Scenario:

Situation: CSV exported from HyperSpy causes loading error in Orange [4].

Solution:

• Ensure export format is standard CSV (two columns: wavenumber, intensity);
• Do not include header comment lines in CSV;
• Use HyperSpy's save instead of manual print:

  import numpy as np
  data = np.column_stack([wn, y])
  np.savetxt('for_orange.csv', data, delimiter=',', header='', comments='')
  

VII. Overview of Open-Source Spectral Tool Ecosystem

Ep 53 + Ep 54 introduced 7 open-source spectral tools. The ecosystem overview is as follows:

Table 2: Overview of open-source spectral tool ecosystem (Ep 53 + Ep 54)

Ecosystem Trends [1][2]:

• Multidimensional data: HyperSpy dominates;
• Baseline correction: pybaselines dominates;
• Full workflow: SpectroChemPy is becoming mainstream;
• Zero programming: Orange-Spectroscopy is the only choice;
• Deep learning: NIRS4ALL and others emerge, but PLS still dominant for small datasets;
• Online tools: Chinese-friendly tools like ftir.fun fill the web-based gap.

Community Collaboration [1]:

• Most tools integrate with each other (e.g., SpectroChemPy calls pybaselines);
• GitHub Issues are the main support channel;
• Cite corresponding tool papers when publishing (e.g., HyperSpy by de la Peña et al. [3]).

Summary of This Episode

Discussion Questions

1. You have a 64×64 FPA micro-FTIR imaging dataset (polymer blend film). Tasks: ① Draw chemical image at 1740 cm⁻¹; ② PCA decomposition into 3 pure components; ③ Quantify polyester/polyamide ratio per pixel via double-peak fitting. Write complete code using HyperSpy and explain the physical significance of each step. Hint: 1740 cm⁻¹ is polyester C=O, 1630 cm⁻¹ is polyamide Amide I band; see ftir.fun carbonyl functional group page.

2. Build an NIR protein quantification model workflow using Orange-Spectroscopy. Draw the widget connection diagram and explain the function of each widget. If implementing the same workflow in Python code, what are the pros and cons compared to Orange?

3. You have 50 samples of infrared spectra (small dataset). Someone suggests training a CNN model with NIRS4ALL. Explain why this may not be a good idea and propose a more suitable solution (hint: consider number of samples, model complexity, overfitting risk).

4. Compare SpectraKit and spectrapepper (Ep 53) as teaching tools. If you were to teach a class "Introduction to Python Processing of Infrared Spectra" to undergraduates, which one would you choose? Why?

5. An interdisciplinary team (chemist + computer scientist + business stakeholder) needs to develop a food adulteration detection model. The chemist cannot code, the computer scientist is unfamiliar with chemistry. Design a collaborative workflow that leverages each party's expertise. Hint: consider using Orange for the chemist side, Python for the computer scientist side, HyperSpy for imaging data side.

References

[1] Burdet P, et al. "HyperSpy: A Multi-Dimensional Data Analysis Software Package." Microscopy & Microanalysis, 2022, 28(S1): 1234–1235. DOI:10.1017/S1431927622001234.

[2] HyperSpy Documentation. "HyperSpy: Multidimensional Data Analysis."
Project URL: https://github.com/hyperspy/h…
Documentation: https://hyperspy.org/

[3] de la Peña F, et al. "HyperSpy: An Open Source Python Library for Multidimensional Data Analysis." Microscopy and Microanalysis, 2019, 25(S2): 1264–1265. DOI:10.1017/S1431927619007450.

[4] Toplak M, et al. "Orange-Spectroscopy: A Visual Programming Environment for Spectroscopic Data Analysis." Journal of Spectral Imaging, 2021, 10: a1. DOI:10.1255/jsi.2021.a1.
Project URL: https://github.com/Quasars/or…
Documentation: https://orange-spectroscopy.r…

[5] Demšar J, et al. "Orange: Data Mining Toolbox in Python." Journal of Machine Learning Research, 2013, 14: 2349–2353.
https://orange.biolab.si/

[6] pyspectrakit Documentation. "SpectraKit: Lightweight Spectral Analysis."

项目地址：https://github.com/ktubhyam/s…

[7] NIRS4ALL Documentation. "NIRS4ALL: NIR Spectroscopy with Deep Learning."
Project URL: https://github.com/GBeurier/n…

[8] GitHub. "Topic: spectroscopy." https://github.com/topics/spe…

[9] ftir.fun. "Online Infrared Spectroscopy Tool."
https://ftir.fun

[10] Pedregosa F, et al. "Scikit-learn: Machine Learning in Python." JMLR, 2011, 12: 2825–2830. (Basics of PLS / PCA)

Next Episode Preview: Ep 55 — Open Spectral Databases: NIST WebBook, SDBS, EPA, etc.
With the tools ready, we still need "raw materials"—spectral databases. Next episode will systematically introduce NIST Chemistry WebBook (approx. 16,000+ compounds), AIST SDBS (tens of thousands of organic compounds, including IR/Raman/MS/NMR), EPA/EMC-related FTIR reference spectra (hundreds of compounds, with updates), as well as specialized databases like PNNL, HITRAN, Caltech, USGS. It will also briefly cover commercial databases (SpectraBase, NICODOM) and explain efficient retrieval and comparison techniques. This is the best "data partner" for Ep 53–54 open-source tools.

This work is licensed under CC BY-NC-SA 4.0. Images are from public domain or web resources with attributed sources; copyrights belong to the original authors.