Hue selectivity from recurrent circuitry

The perception of color involves a transformation from the spectral properties of visual stimuli to derived perceptual quantities such as brightness, saturation and hue. Although hue selective neurons, which respond to narrow regions in color space, have been reported in primates, they have not been identified in other species including more accessible organisms, which would facilitate circuit level analyses. Here we show that neurons in the Drosophila optic lobe have hue selective properties, with narrow tuning for both spectral and non-spectral colors. We construct a connectomics constrained circuit model that accounts for this hue selectivity. Our model, combined with genetic manipulations, shows that recurrent connections in the circuit are critical for the tuning properties of Drosophila hue selective neurons. Our findings reveal the circuit basis for a transition from physical detection to sensory perception in color vision.

Hue selectivity from recurrent circuitry, Nature Neuroscience, 2024

Presented at Columbia Neurotheory Meeting, 2022

Circuit mechanisms of chromatic encoding

Spectral information is commonly processed in the brain through generation of antagonistic responses to different wavelengths. In many species, these color opponent signals arise as early as photoreceptor terminals. Here, we measure the spectral tuning of photoreceptors in Drosophila. In addition to a previously described pathway comparing wavelengths at each point in space, we find a horizontal-cell-mediated pathway similar to that found in mammals. This pathway enables additional spectral comparisons through lateral inhibition, expanding the range of chromatic encoding in the fly. Together, these two pathways enable efficient decorrelation and dimensionality reduction of photoreceptor signals while retaining maximal chromatic information. A biologically constrained model accounts for our findings and predicts a spatio-chromatic receptive field for fly photoreceptor outputs, with a color opponent center and broadband surround. This dual mechanism combines motifs of both an insect-specific visual circuit and an evolutionarily convergent circuit architecture, endowing flies with the ability to extract chromatic information at distinct spatial resolutions.

Circuit Mechanisms Underlying Chromatic Encoding in Drosophila Photoreceptors, Current Biology, 2020

Presented at Columbia Neurotheory Meeting, 2019

Normative models of spatio-spectral decorrelation

In line with the efficient coding hypothesis, the early visual system aims to minimize spectral and spatial redundancies arising from overlapping opsin sensitivities in retinal photoreceptors (PRs) and highly correlated structure in natural scenes. Encoding color information, or spectral information independent of intensity, requires comparing activities across different types of PRs. Mounting evidence shows that several species across the animal kingdom, such as the fruit fly Drosophila Melanogaster, have an uneven proportion of PR types in their retinas. However, it is unknown whether this uneven proportion is optimized for objectives relevant to the early color processing of natural scenes, as previous studies have modeled spectral and spatial processing in the early fly visual system independently. We built a collection of models incorporating both spatial and spectral information to solve tasks relevant to the fly’s early visual system, such as predictive coding at the level of PR inputs for spatial decorrelation in the retina as well as spatial and spectral decorrelation at the level of PR outputs via color opponency mechanisms. Using this framework, we asked how varying the ratio of the fly’s two main PR types changed performance accuracy on these tasks. From this normative approach, we were able to conclude that the optimal ratio of PR types to best solve these tasks aligns with the experimentally observed distribution and showed this for multiple opsin sensitivity profiles determined within and across labs. Moreover, shuffling either spatial or spectral information in the input natural scene predicted an even PR type ratio, suggesting that biologically observed PR type ratios are optimized for spectral and spatial decorrelation. Altogether, these results suggest that natural scene statistics may have shaped the ratio of PR types in the fly retina through evolutionary mechanisms, providing important implications for understanding sensory systems in an ecologically relevant context.

Normative models of spatio-spectral decorrelation predict observed receptor distributions, CoSyNe, 2022

Geometry of color spaces (dreye)

Color vision represents a vital aspect of perception that ultimately enables a wide variety of species to thrive in the natural world. However, unified methods for constructing chromatic visual stimuli in a laboratory setting are lacking. Here, we present stimulus design methods and an accompanying programming package to efficiently probe the color space of any species in which the photoreceptor spectral sensitivities are known. Our hardware-agnostic approach incorporates photoreceptor models within the framework of the principle of univariance. This enables experimenters to identify the most effective way to combine multiple light sources to create desired distributions of light, and thus easily construct relevant stimuli for mapping the color space of an organism. We include methodology to handle uncertainty of photoreceptor spectral sensitivity as well as to optimally reconstruct hyperspectral images given recent hardware advances. Our methods support broad applications in color vision science and provide a framework for uniform stimulus designs across experimental systems.

Exploiting colour space geometry for visual stimulus design across animals, Phil. Trans. R. Soc. B, 2022

neuralsignal/dreye

Probabilistic circuit model analysis

Sensory circuits are complex systems that process information through feedforward and recurrent interactions among a set of neurons. While these circuits have been studied extensively, modeling their underlying mechanisms and computations can be challenging when not all neurons in the circuit are recorded from and when synapse information is incomplete or inaccurate. In this study, we propose a probabilistic framework that predicts the responses of unobserved neurons using anatomical constraints. We demonstrate the effectiveness of our approach on both simulated data using randomly generated connectomes with varying complexities and real neural data from the fruit fly optic lobe. Our approach accurately predicts the responses of unobserved neurons in simulated and real data. Given our probabilistic framework, we are also able to infer what types of experiments are optimal for subsequent experiments. Overall, our method provides a useful tool for modeling the complex mechanisms underlying sensory information processing in biological circuits.

Presented at Columbia Neurotheory Meeting, 2020 and 2021

neuralsignal/scidoggo

Encoding models (scidoggo)

This coding project is a collection of different models and data tools that I have created, modified, and used for my various research projects. Included in the package are various types of constraint convex optimization procedures, interpolation models, nonlinear encoding models, and probabilistic models for circuit model analysis. The project is still under development and can be found on GitHub.

neuralsignal/scidoggo

Biometric time-series models for healthcare monitoring

In underserved communities, elderly individuals often lack access to advanced health monitoring that could detect early signs of common issues like infections, pneumonia, and constipation. These conditions can be identified through multi-day patterns in data from wearable devices. While machine learning can enable analysis of such long-term biometric data, typical models struggle with very long input sequences. Recently developed deep state-space models overcome this limitation and scale linearly with sequence length. This project will develop a tailored deep state-space model architecture to learn individual “biometric profiles” and detect healthcare events. It will also build software tools to enable training and deployment of the models to expand access to sophisticated healthcare monitoring.

Field Casualty Management AI

In a near-peer conflict, warfighters may be in the field without support for days to weeks. This is especially problematic for managing casualties. In such scenarios, rationing limited quantities of blood or antibiotics requires predicting the individual progression of blood loss, or the likelihood of sepsis. While military medics are highly skilled at initial treatment of traumatic injuries, they are not trained to track the health trends of wounded warfighters over extended periods. Blood loss and sepsis produce typified patterns in heart rate, blood oxygen and other biometrics commonly collected from wearable devices. These patterns are individual-specific and emerge over hours to days. Recent advances in machine learning methods are only now making detecting such patterns possible. We are developing a generalized artificial intelligence (AI) for interpreting individual-specific, long-timescale biophysical data from wearable devices. While the AI can be used to detect and track a wide variety of health conditions, we are particularly focused on managing warfighter casualties in the field. The AI integrates into commonly used hardware and software, acting as a “force multiplier,” for existing equipment. Our AI will aid medics manage field casualties, saving lives when warfighters are cut off from advanced medical care.

Field casualty management AI, W.W. Pettine, M. Christenson, P. Koirala, Defense TechConnect, 2023
Assessing Foundation Models’ Transferability to Physiological Signals in Precision Medicine, M. Christenson, C. Geary, B. Locke, P. Koirala, W.W. Pettine, Defense AI in Medicine, 2024

Agentic engineering kit

A composable library of engineering standards, automated workflows, and reusable components for AI-assisted software development. Includes principles, on-demand skills, and autonomous GitHub Actions that can be integrated via AI assistant rules, git subtree, or individual files. Designed as a governance framework that teams can adapt to their specific workflows.

neuralsignal/agentic-engineering-kit

Wikipedia to YouTube shorts

In this proof of concept, I am scraping the Wikipedia article of the day to create youtube shorts from start to finish using OpenAI’s GPT model and various open-source HuggingFace models to convert text to audio, text to images, and text to video. The video encoding needs to be improved and the images aren’t nicely cut into a smooth sequence.

neuralsignal/wikioftheday

Movie database querying

In this proof of concept, I am trying to play around with various LLM APIs, including langchain and llamaindex, to build a semantic search over a movie database.

neuralsignal/movie_playground

Auto-GPT contributions

I contributed a couple of features for this project, mainly an audio-to-text converter and image summarizer.

Significant-Gravitas/Auto-GPT

Obsidian export

Converts Obsidian-flavored Markdown into professional PDF and DOCX documents through a 5-stage processing pipeline. Handles wikilinks, embeds, Mermaid diagrams, and callouts. Supports customizable profiles for branded styling.

neuralsignal/obsidian-export

Obsidian import

The inverse — extracts content from PDFs, Word documents, PowerPoint, spreadsheets, and other formats into Obsidian-ready Markdown with YAML frontmatter. Supports batch processing with multiple extraction backends.

neuralsignal/obsidian-import

Excel financial model tooling

excel-model converts YAML specifications into professional Excel workbooks with formulas, styling, and named ranges. It supports multiple model types including P&L statements, DCF analyses, budgets, and scenario comparisons with 21 built-in formula types for financial calculations. Available as both a CLI tool and Python API.

neuralsignal/excel-model

Loris

Loris is a database and analysis application that is currently under development. The project is being tested, and core features are being added. Once the core features are stable, documentation for the different features will be added. Loris was originally created to provide a database system for the Behnia lab. The lab uses it for data entry, maintenance, and analysis on a daily basis. While the primary use case is for a Drosophila lab, it is flexible enough to be used by other research labs as well.

neuralsignal/loris

Puffbird

Puffbird is an open-source library that extends the functionality of pandas DataFrames by providing a user-friendly interface for handling complex data structures with multiple nested cells. The library has been developed with a focus on ease of use and follows the documentation style of pandas. While some of the features in puffbird have become redundant with the addition of new functionalities such as the explode method in pandas, the library continues to provide a valuable extension to pandas. The project is actively maintained, albeit at an irregular rate, and welcomes contributions from the community.

neuralsignal/puffbird