Avalonia with Fragment Shaders
A comprehensive guide to using fragment shaders with Avalonia
A comprehensive guide to using fragment shaders with Avalonia
Avalonia is a powerful cross-platform UI framework that, when combined with fragment shaders, opens up a world of possibilities for creating stunning visual effects in your applications. This comprehensive guide will explore how to use shaders in Avalonia, focusing on creating a custom control that can render and animate them. We'll start with the basics and build our way up to a fully functional ShaderAnimationControl.
Before we dive into the implementation, let's review some key concepts:
To follow this guide, ensure your project uses Avalonia 11.2.1 or newer. We'll assume you have basic knowledge of Avalonia and fragment shaders moving forward.
Shadertoy is a popular platform for creating and sharing exciting and visually captivating fragment shaders. To use these shaders with Avalonia, we need to convert them to SkSL. The conversion process involves several key steps:
Uniform declarations:
vec3 to float3) depending on your SkiaSharp version.Main function signature:
void mainImage(out vec4 fragColor, in vec2 fragCoord) to vec4 main(vec2 fragCoord)fragColor at the end of the function instead of using an out parameter.Adjust uniform and variable types:
vec2, vec3, vec4 to float2, float3, float4 respectively.int uniforms with float (e.g., uniform int iFrame becomes uniform float iFrame). Cast to int in the shader when needed: int currentFrame = int(iFrame);Update coordinate system:
fragCoord.y = iResolution.y - fragCoord.y;
Normalize coordinates:
float2 uv = fragCoord/iResolution.xy;
Here's an example of the default Shadertoy shader and its SkSL conversion:
Original Shadertoy shader:
uniform vec3 iResolution; // viewport resolution (in pixels)
uniform float iTime; // shader playback time (in seconds)
uniform float iTimeDelta; // render time (in seconds)
uniform float iFrameRate; // shader frame rate
uniform int iFrame; // shader playback frame
uniform float iChannelTime[4]; // channel playback time (in seconds)
uniform vec3 iChannelResolution[4]; // channel resolution (in pixels)
uniform vec4 iMouse; // mouse pixel coords. xy: current (if MLB down), zw: click
void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
// Normalized pixel coordinates (from 0 to 1)
vec2 uv = fragCoord/iResolution.xy;
// Time varying pixel color
vec3 col = 0.5 + 0.5*cos(iTime+uv.xyx+vec3(0,2,4));
// Output to screen
fragColor = vec4(col,1.0);
}
Converted SkSL version:
uniform float2 iResolution;
uniform float iTime;
float4 main(float2 fragCoord)
{
// Flip y-coordinate
fragCoord.y = iResolution.y - fragCoord.y;
// Normalized pixel coordinates (from 0 to 1)
float2 uv = fragCoord/iResolution.xy;
// Time varying pixel color
float3 col = 0.5 + 0.5*cos(iTime+float3(uv,0)+float3(0,2,4));
// Output to screen
return float4(col,1.0);
}
Key differences and changes:
void mainImage(out vec4 fragColor, in vec2 fragCoord) to float4 main(float2 fragCoord).vec3 to float3, vec4 to float4, int to float).fragColor out parameter, we now return it at the end of the function.float2 and float3 instead of vec2 and vec3 in the shader code.iResolution from float3 to float2 to account for the fact that the control is only going to pass width and height values.When using these shaders with the ShaderAnimationControl, you'll need to ensure that you're passing the correct uniforms from your C# code. The ShaderAnimationControl we're building will handle passing iResolution and iTime. If your shader uses other uniforms like iMouse, you'll need to extend the control to pass these values as well.
Important Notes
Remember that some more complex fragment shaders may require additional modifications and that this article is not exhaustive of all the modifications you need to make, especially if they use features not directly supported in SkSL.
Always test your converted shaders thoroughly and be prepared to make additional adjustments as needed.
Please be mindful of the licensing of any shader code while converting and using them from Shadertoy or other similar websites & sources to your project. Make sure to properly attribute the author and compensate them according to the licensing terms of the said shader code.
The ShaderAnimationControl is a custom Avalonia control that allows for the rendering and animation of fragment shaders. We'll be building it step by step to understand its structure and functionality.
Let's define the ShaderAnimationControl class and its private structures as a starting point:
using Avalonia;
using Avalonia.Controls;
using Avalonia.Media;
using Avalonia.Platform;
using Avalonia.Rendering.Composition;
using Avalonia.Skia;
using SkiaSharp;
using System;
using System.Diagnostics;
using System.IO;
using System.Numerics;
public class ShaderAnimationControl : UserControl
{
private record struct ShaderDrawPayload(
HandlerCommand HandlerCommand,
Uri? ShaderCode = default,
Size? ShaderSize = default,
Size? Size = default,
Stretch? Stretch = default,
StretchDirection? StretchDirection = default);
private enum HandlerCommand
{
Start,
Stop,
Update,
Dispose
}
private const float DefaultShaderLength = 512;
private CompositionCustomVisual? _customVisual;
// ... (rest of the code will be explained in subsequent sections)
}
This structure sets up the foundation for the ShaderAnimationControl. The ShaderDrawPayload record struct and HandlerCommand enum work together to facilitate communication between the main control and its rendering logic in _customVisual.
The use of private nested types (the record struct and enum) encapsulates these implementation details within the ShaderAnimationControl class.
The ShaderAnimationControl class defines several Avalonia properties to allow customization of the shader's appearance and behavior.
public class ShaderAnimationControl : UserControl
{
public static readonly StyledProperty<Stretch> StretchProperty =
AvaloniaProperty.Register<ShaderAnimationControl, Stretch>(nameof(Stretch), Stretch.Uniform);
public static readonly StyledProperty<StretchDirection> StretchDirectionProperty =
AvaloniaProperty.Register<ShaderAnimationControl, StretchDirection>(
nameof(StretchDirection),
StretchDirection.Both);
public static readonly StyledProperty<Uri?> ShaderUriProperty =
AvaloniaProperty.Register<ShaderAnimationControl, Uri?>(nameof(ShaderUri));
public static readonly StyledProperty<double> ShaderWidthProperty =
AvaloniaProperty.Register<ShaderAnimationControl, double>(
nameof(ShaderWidth),
DefaultShaderLength);
public static readonly StyledProperty<double> ShaderHeightProperty =
AvaloniaProperty.Register<ShaderAnimationControl, double>(
nameof(ShaderHeight),
DefaultShaderLength);
public Stretch Stretch
{
get => GetValue(StretchProperty);
set => SetValue(StretchProperty, value);
}
public StretchDirection StretchDirection
{
get => GetValue(StretchDirectionProperty);
set => SetValue(StretchDirectionProperty, value);
}
public Uri? ShaderUri
{
get => GetValue(ShaderUriProperty);
set => SetValue(ShaderUriProperty, value);
}
public double ShaderWidth
{
get => GetValue(ShaderWidthProperty);
set => SetValue(ShaderWidthProperty, value);
}
public double ShaderHeight
{
get => GetValue(ShaderHeightProperty);
set => SetValue(ShaderHeightProperty, value);
}
static ShaderAnimationControl()
{
AffectsRender<ShaderAnimationControl>(ShaderUriProperty,
StretchProperty,
StretchDirectionProperty,
ShaderWidthProperty,
ShaderHeightProperty);
AffectsMeasure<ShaderAnimationControl>(ShaderUriProperty,
StretchProperty,
StretchDirectionProperty,
ShaderWidthProperty,
ShaderHeightProperty);
}
}
Let's break down each property and its purpose:
StretchProperty: Determines how the shader content is stretched to fill the control's bounds.
StretchDirectionProperty: Specifies in which directions the shader content can be stretched.
ShaderUriProperty: Holds the URI of the shader code file to be loaded and rendered. In this control's case, it should be in avares:// format and the shader must be an Avalonia Resource. This is explained further in section Using the ShaderAnimationControl in Your Avalonia Project.
ShaderWidthProperty and ShaderHeightProperty: Defines the width and height of the shader's rendering area. Defaults to 512 x 512 when not set.
The static constructor of the class uses AffectsRender and AffectsMeasure to indicate that changes to these properties should trigger re-rendering and re-measurement of the control. This ensures that the control updates appropriately when these properties change.
The control overrides methods for measuring and arranging itself:
private Size GetShaderSize()
{
return new Size(ShaderWidth, ShaderHeight);
}
protected override Size MeasureOverride(Size availableSize)
{
var source = ShaderUri;
var result = new Size();
if (source != null)
{
result = Stretch.CalculateSize(availableSize, GetShaderSize(), StretchDirection);
}
return result;
}
protected override Size ArrangeOverride(Size finalSize)
{
var source = ShaderUri;
if (source == null) return new Size();
var sourceSize = GetShaderSize();
var result = Stretch.CalculateSize(finalSize, sourceSize);
return result;
}
These methods ensure that the control sizes itself correctly based on the shader size and stretch settings.
The ShaderAnimationControl overrides methods for attaching to and detaching from the visual tree. These methods are crucial for initializing the custom visual when the control is added to the UI and cleaning up resources when it's removed.
protected override void OnAttachedToVisualTree(VisualTreeAttachmentEventArgs e)
{
base.OnAttachedToVisualTree(e);
var elemVisual = ElementComposition.GetElementVisual(this);
var compositor = elemVisual?.Compositor;
if (compositor is null)
{
return;
}
_customVisual = compositor.CreateCustomVisual(new ShaderCompositionCustomVisualHandler());
ElementComposition.SetElementChildVisual(this, _customVisual);
LayoutUpdated += OnLayoutUpdated;
_customVisual.Size = new Vector2((float)Bounds.Size.Width, (float)Bounds.Size.Height);
_customVisual.SendHandlerMessage(
new ShaderDrawPayload(
HandlerCommand.Update,
null,
GetShaderSize(),
Bounds.Size,
Stretch,
StretchDirection));
Start();
}
protected override void OnDetachedFromVisualTree(VisualTreeAttachmentEventArgs e)
{
base.OnDetachedFromVisualTree(e);
LayoutUpdated -= OnLayoutUpdated;
Stop();
DisposeImpl();
}
private void OnLayoutUpdated(object? sender, EventArgs e)
{
if (_customVisual == null)
{
return;
}
_customVisual.Size = new Vector2((float)Bounds.Size.Width, (float)Bounds.Size.Height);
_customVisual.SendHandlerMessage(
new ShaderDrawPayload(
HandlerCommand.Update,
null,
GetShaderSize(),
Bounds.Size,
Stretch,
StretchDirection));
}
Let's break down some of the key parts of the OnAttachedToVisualTree method:
var elemVisual = ElementComposition.GetElementVisual(this);
var compositor = elemVisual?.Compositor;
if (compositor is null)
{
return;
}
_customVisual = compositor.CreateCustomVisual(new ShaderCompositionCustomVisualHandler());
ElementComposition.SetElementChildVisual(this, _customVisual);
This block of code is responsible for setting up the composition visual elements:
ElementVisual for the control and its associated Compositor.CustomVisual using ShaderCompositionCustomVisualHandler.This code is crucial for enabling high-performance rendering of the shader, as it allows the shader to be rendered independently of the main UI thread.
The control subscribes to the LayoutUpdated event to handle layout changes. Then the custom visual's size is set based on the control's bounds, and an initial update message is sent to the handler. The Start() method is then called to begin the shader animation.
The OnDetachedFromVisualTree method performs cleanup operations:
LayoutUpdated event.These methods ensure that the ShaderAnimationControl properly initializes when added to the UI and cleans up when removed, preventing resource leaks and ensuring proper integration with Avalonia's visual tree management.
Lastly, the OnLayoutUpdated method sends the latest layout bounds to the custom visual handler.
The class includes methods for starting, stopping, and disposing of the shader:
private void Start()
{
_customVisual?.SendHandlerMessage(
new ShaderDrawPayload(
HandlerCommand.Start,
ShaderUri,
GetShaderSize(),
Bounds.Size,
Stretch,
StretchDirection));
InvalidateVisual();
}
private void Stop()
{
_customVisual?.SendHandlerMessage(new ShaderDrawPayload(HandlerCommand.Stop));
}
private void DisposeImpl()
{
_customVisual?.SendHandlerMessage(new ShaderDrawPayload(HandlerCommand.Dispose));
}
These methods send appropriate messages to the custom visual handler to control the shader's lifecycle.
The ShaderCompositionCustomVisualHandler is a class defined inside ShaderAnimationControl that is responsible for managing the shader's lifecycle, handling animation frame updates, and performing the actual fragment shader rendering using SkiaSharp and Composition API's. Let's break it down further:
It is based around the CompositionCustomVisualHandler class and Avalonia's Composition API's that provides mechanisms to process custom visual rendering that is independent of the app's UI Thread.
This allows for better performance and visual responsiveness and also allows for some kind of visual feedback to the user if the UI Thread is blocked for some reason.
public class ShaderAnimationControl : UserControl
{
// ... (rest of the prior code here)
private class ShaderCompositionCustomVisualHandler : CompositionCustomVisualHandler
{
private bool _running;
private Stretch? _stretch;
private StretchDirection? _stretchDirection;
private Size? _boundsSize;
private Size? _shaderSize;
private string? _shaderCode;
private readonly object _sync = new();
private SKRuntimeEffectUniforms? _uniforms;
private SKRuntimeEffect? _effect;
private bool _isDisposed;
// ... (methods will be explained in subsequent sections)
}
}
Let's break down these fields:
_running: A boolean flag indicating whether the shader animation is currently running._stretch and _stretchDirection: These fields store the current stretch settings for the shader._boundsSize: Represents the current size of the control's bounds._shaderSize: Stores the size of the shader itself._shaderCode: Holds the actual GLSL code of the shader._sync: An object used for thread synchronization._uniforms: An instance of SKRuntimeEffectUniforms used to pass uniform values to the shader._effect: An instance of SKRuntimeEffect representing the compiled shader._isDisposed: A flag indicating whether the handler has been disposed.These fields are crucial for managing the state of the shader and ensuring proper rendering and lifecycle management. The use of nullable types (e.g., Stretch?) allows for cases where these values might not be set.
The OnMessage method is responsible for handling messages sent from the main control:
public override void OnMessage(object message)
{
if (message is not ShaderDrawPayload msg)
{
return;
}
switch (msg)
{
case { HandlerCommand: HandlerCommand.Start, ShaderCode: { } uri, ShaderSize: { } shaderSize, Size: { } size, Stretch: { } st, StretchDirection: { } sd }:
{
using var stream = AssetLoader.Open(uri);
using var txt = new StreamReader(stream);
_shaderCode = txt.ReadToEnd();
_effect = SKRuntimeEffect.Create(_shaderCode, out var errorText);
if (_effect == null)
{
Console.WriteLine($"Shader compilation error: {errorText}");
}
_shaderSize = shaderSize;
_running = true;
_boundsSize = size;
_stretch = st;
_stretchDirection = sd;
RegisterForNextAnimationFrameUpdate();
break;
}
case { HandlerCommand: HandlerCommand.Update, ShaderSize: { } shaderSize, Size: { } size, Stretch: { } st, StretchDirection: { } sd }:
{
_shaderSize = shaderSize;
_boundsSize = size;
_stretch = st;
_stretchDirection = sd;
RegisterForNextAnimationFrameUpdate();
break;
}
case { HandlerCommand: HandlerCommand.Stop }:
{
_running = false;
break;
}
case { HandlerCommand: HandlerCommand.Dispose }:
{
DisposeImpl();
break;
}
}
}
This method handles different commands:
Start: Loads and compiles the shader, sets up initial parameters, and starts the animation.Update: Updates the shader's size and stretch parameters.Stop: Stops the animation.Dispose: Cleans up resources.The OnAnimationFrameUpdate method is called for each animation frame:
public override void OnAnimationFrameUpdate()
{
if (!_running || _isDisposed)
return;
Invalidate();
RegisterForNextAnimationFrameUpdate();
}
This method invalidates the visual (triggering a redraw) and registers for the next frame update, creating a continuous animation loop.
The Draw method is responsible for actually rendering the shader:
private void Draw(SKCanvas canvas)
{
if (_isDisposed || _effect is null)
return;
canvas.Save();
var targetWidth = (float)(_shaderSize?.Width ?? DefaultShaderLength);
var targetHeight = (float)(_shaderSize?.Height ?? DefaultShaderLength);
_uniforms ??= new SKRuntimeEffectUniforms(_effect);
_uniforms["iTime"] = (float)CompositionNow.TotalSeconds;
_uniforms["iResolution"] = new[] { targetWidth, targetHeight };
using (var paint = new SKPaint())
using (var shader = _effect.ToShader(false, _uniforms))
{
paint.Shader = shader;
canvas.DrawRect(SKRect.Create(targetWidth, targetHeight), paint);
}
canvas.Restore();
}
This method sets up the shader uniforms (including time for animation) and draws the shader onto the canvas. This is the place where you can add more uniforms to the shader as needed.
The OnRender method is called when the control needs to be redrawn:
public override void OnRender(ImmediateDrawingContext context)
{
lock (_sync)
{
if (_stretch is not { } st
|| _stretchDirection is not { } sd
|| _isDisposed)
{
return;
}
var leaseFeature = context.TryGetFeature<ISkiaSharpApiLeaseFeature>();
if (leaseFeature is null)
{
return;
}
var rb = GetRenderBounds();
var size = _boundsSize ?? rb.Size;
var viewPort = new Rect(rb.Size);
var sourceSize = _shaderSize!.Value;
if (sourceSize.Width <= 0 || sourceSize.Height <= 0)
{
return;
}
var scale = st.CalculateScaling(rb.Size, sourceSize, sd);
var scaledSize = sourceSize * scale;
var destRect = viewPort
.CenterRect(new Rect(scaledSize))
.Intersect(viewPort);
var sourceRect = new Rect(sourceSize)
.CenterRect(new Rect(destRect.Size / scale));
var bounds = SKRect.Create(new SKPoint(), new SKSize((float)size.Width, (float)size.Height));
var scaleMatrix = Matrix.CreateScale(
destRect.Width / sourceRect.Width,
destRect.Height / sourceRect.Height);
var translateMatrix = Matrix.CreateTranslation(
-sourceRect.X + destRect.X - bounds.Top,
-sourceRect.Y + destRect.Y - bounds.Left);
using (context.PushClip(destRect))
using (context.PushPostTransform(translateMatrix * scaleMatrix))
{
using var lease = leaseFeature.Lease();
var canvas = lease.SkCanvas;
Draw(canvas);
}
}
}
This method handles:
Draw method to render the shader on the canvas.private void DisposeImpl()
{
lock (_sync)
{
if (_isDisposed) return;
_isDisposed = true;
_effect?.Dispose();
_uniforms?.Reset();
_running = false;
}
}
This ensures that the SkiaSharp assets in the custom visual handler are properly disposed of, that the handler stops playback, and avoids memory leaks.
Now that we've built our ShaderAnimationControl, let's see how you can use it in your Avalonia project.
Add the control to your XAML:
<Window xmlns:controls="using:YourNamespace.Controls">
<!-- other content -->
<controls:ShaderAnimationControl x:Name="ShaderHost"
ShaderUri="avares://YourProject/Assets/your_shader.sksl"/>
</Window>
Ensure your shader file is included in your project:
(Project Root Folder)/Assets then make sure that the project file has the following entry:
<AvaloniaResource Include="Assets\**" />
Customize the control:
Stretch, StretchDirection, ShaderWidth, and ShaderHeight properties as needed.Handle playback events:
protected override void OnLoaded(RoutedEventArgs e)
{
base.OnLoaded(e);
ShaderHost.Start();
}
protected override void OnUnloaded(RoutedEventArgs e)
{
base.OnUnloaded(e);
ShaderHost.Stop();
}
With the ShaderAnimationControl, you can now easily incorporate stunning shader effects into your Avalonia app. This control provides a flexible way to load, render, and animate fragment shaders, opening up a world of creative possibilities for your app's visuals.
Remember to optimize your shaders for performance, especially on mobile devices or when using complex effects. The ShaderAnimationControl provides a solid foundation for shader integration, but you may need to extend it further for more complex use cases, such as passing additional uniforms or handling user interactions.
As you experiment with different shaders and effects, keep in mind the balance between visual appeal and performance. Complex shaders can be resource-intensive, so always test your application on various devices to ensure a smooth user experience.
Happy coding, and enjoy creating captivating visual effects with Avalonia and fragment shaders!
Special thanks to Wiesław Šoltés for giving us the permission to use his EffectsDemo project as an inspiration to the article's ShaderAnimationControl code.
Here’s what you might have missed.