mirror of
https://git.ryujinx.app/ryubing/ryujinx.git
synced 2026-06-26 14:19:04 +00:00
8b1b015572
Ayyyy, welcome to the **macOS: Fix Hypervisor BadArgument Launch Crash PR!**
We are so so back my friends.
This PR fixes the macOS Hypervisor `BadArgument` launch crash, while also sneaking in some improvements to help _potentially_ mitigate those lovely 0 FPS deadlock on macOS in certain games with Vulkan + Hypervisor enabled (there will be a separate PR in the future to address this).
The PR is slightly different in terms of the PR messages that I write, but I hope that it'll provide the necessary context to those unaware of the reasoning/causes of the issue.
Also, this is a bit of a read, so grab some snacks, pour a drink, get comfortable, and enjoy some…
### STORY TIME!!!
_Before we proceed forward, we must properly understand what Hypervisor is._
_Hypervisor refers to using Apple's built-in virtualization technology, the Hypervisor Framework, to accelerate CPU emulation._
_There are two main ways to do this:_
_1. **Interpreter** – executes instructions one by one (very slow)._
_2. **JIT (Just-In-Time)** recompilation – translates Switch CPU instructions into native code (much faster)._
_On Apple Silicon Macs (M1, M2, M3, etc.), the guest CPU and host CPU are both ARM64-based. As the Nintendo Switch itself is ARM64, The Hypervisor Framework can allow Ryujinx to execute large amounts of guest code more directly, reducing emulation overhead. The architectural similarity makes hardware-assisted execution practical. On Intel Macs, there isn't the same direct ARM-to-ARM advantage, so the benefits are much more limited or unavailable._
_When Hypervisor mode is available and working correctly, it can provide higher frames, reduced CPU bottlenecks, faster shader compilation in certain instances, smoother gameplay, and lower CPU usage._
_The improvement depends heavily on the game. CPU-heavy titles tend to benefit the most._
_There are downsides though. Sometimes, certain games may have compatibilities issues, certain feature may not work exactly the same, bugs in the actual Hypervisor implementation can cause crashes or issues, and performance gain vary significantly on a game-to-game basis._
_In terms of a simple analogy:_
_**Without Hypervisor:** Ryujinx acts like a translator who rewrites every sentence before speaking it._
_**With Hypervisor:** Ryujinx can let the Mac's CPU understand and execute much more of the Switch's code directly, so less translation work is needed._
_Anyway, now that we have the background, onto the story!_
Once upon a time, around ~December 2025 / January 2026, users — and yes, even devs (though let’s be honest, macOS devs are basically cryptids at this point)
_(Authors note: Cryptids - animals or other beings whose present existence is disputed or unsubstantiated by science)_
Started noticing inconsistent behaviour with the macOS Hypervisor. In other words, Hypervisor began showing inconsistent launch behaviour for users who've upgraded to the then current macOS versions (15.7.3 / 26.2). This behaviour continued as macOS got updated, and is still present on the latest version (Tahoe 26.5.1).
Initially, Hypervisor was mostly crashing with an `Unexpected result "Denied"`. The likely cause is two fold:
1. macOS security permissions changed during that update.
2. Hypervisor framework got updated, but nothing what communicated to the devs.
(It is likely to be the former, not the later. This is the main theory, at least.)
However, this crash later evolved into (because of course it did) into `Unexpected result "BadArgument"`.
We'll come back to this in a bit.
For users who wished to run games with Hypervisor enabled, they had a simple solution - build Ryujinx locally. Yes, this is an effective solution in its own right, as that does bypass certain macOS signing and certain .app bundle-related constraints, as well as certain security permissions.
However, this doesn't explain why:
* Official server builds were the only builds that were affected
* Users on _older macOS_ (below 15.7.3 / 26.2) could run games with Hypervisor enabled as if nothing had ever gone wrong in their lives.
* Users on _newer macOS_ versions _could still run certain older_ Ryujinx builds as intended — specifically, the last or close-to-last version they had lying around in their Downloads folder.
* Users on macOS above 15.7.3 / 26.2 could still run games with Hypervisor enabled, on builds downloaded from the server, for whatever reason.
So now we’re here:
* Same macOS versions → Different outcomes
* Different Ryujinx builds → Different behavior
* `Denied` → Evolved into `BadArgument`
* No consistent reproduction steps
And so, after attempting to fix this for quite some time, I've determined it to be unfixable. Ryujinx should drop macOS support because development has been held back far too long, and there is nothing we can possibly do. Every possible solution was tested and the results were null.
So, it is with great sadness that, here and now, I proclaim that Ryujinx will terminate further macOS support.
Bye-bye, Ryujinx on macOS! You had a great run, and you'll always be remembered. Thank you for bringing so much great memories to us, macOS users!
...
...
...
...
...
HAHA - NOT SO FAST!
(_Please tell me I got you there_)
Not all hope is lost - there is light at the end of the tunnel!
**The key insight is this:** `BadArgument` is not just “`Denied` but newer and worse”. It’s something else entirely.
* `Denied` → macOS refuses the request at the security boundary. Nothing even starts.
* `BadArgument` → macOS accepts the request, but the data being passed is now invalid under newer constraints.
So instead of:
_“You shall not pass.”_
We now have:
_“You may pass… but what is THAT?”_
And this brings us to this PR!
### The PR
Users on macOS were experiencing two major issues when Hypervisor was enabled:
**1. Hard Crash – "Unexpected result `BadArgument`"**
The most frequent stack trace pointed to:
```
System.Exception: Unexpected result "BadArgument".
at Ryujinx.Cpu.AppleHv.HvResultExtensions.ThrowOnError(...)
at Ryujinx.Cpu.AppleHv.HvExecutionContextVcpu.GetX(Int32 index)
at Ryujinx.HLE.HOS.Kernel.Threading.KThread.GetCurrentContext()
at Ryujinx.HLE.HOS.Kernel.Threading.KThread.GetThreadContext3(...)
```
This crash occurred very early during game initialization, most often inside supervisor calls such as `GetThreadContext3`.
**2. Intermittent Permanent 0 FPS Freezes with Vulkan (unrelated to `BadArgument`, but somewhat addressed in this PR)**
The game would run normally for a while, then drop to 0 FPS permanently. Disabling Hypervisor eliminated the freezes but performance was at times inconsistent.
### The Explanation
`HvResult.BadArgument` is returned by Apple's Hypervisor framework (`hv_vcpu_get_reg`, `hv_vcpu_get_sys_reg`, `hv_vcpu_get_simd_fp_reg`, etc.) when it determines that the requested operation cannot be performed in the current vCPU state.
**Common reasons why this happens:**
* Timing races - the guest tries to read registers before the vCPU has fully synchronized its internal state after context switches, exceptions, or vCPU pool reuse.
* Stricter validation in newer macOS versions.
* vCPU state inconsistencies during early boot or heavy syscall activity.
Because the original implementation called `.ThrowOnError()` unconditionally on every Hypervisor call, even a single transient `BadArgument` would immediately terminate execution. This was the direct cause of the crash.
The 0 FPS issue was a separate but related symptom: when Hypervisor is enabled, the guest CPU runs significantly faster than MoltenVK's presentation queue expects, leading to command buffer starvation and queue deadlock.
### The Changes
* **Implemented: Complete Shadow Register Cache (`HvExecutionContextVcpu.cs`)**
* Added full in-memory shadow copies of every relevant register:
* General-purpose registers: `private readonly ulong[] _x = new ulong[32]`;
* Vector/SIMD registers: private readonly `V128[] _v = new V128[32]`;
* System/special registers: `_pc`, `_elrEl1`, `_esrEl1`, `_tpidrEl0`, `_tpidrroEl0`, `_fpcr`, `_fpsr`, and `_pstateRaw` (stored as `ulong` internally to avoid type conversion issues).
* **Reasoning:** Returning zero or garbage values on failure could corrupt game state. When the Hypervisor returns `BadArgument`, we now have a safe, previously-valid value to return instead of crashing. A shadow cache gives us that last known good value. The cache is updated on every successful read, so correctness is preserved in normal operation. This is the central fix for the `BadArgument` crash.
* **Defensive BadArgument Handling on Every Register Access Path**
* Updated every read operation (`GetX`, `GetV`, `Pstate`, cached getters, etc.) to explicitly check the return value, with the same pattern applied across all accessors.
* **Reasoning:** This directly eliminates the crash while preserving the original error-throwing behavior for genuine failures. Returning cached data is far safer than injecting zero or garbage values that could corrupt guest state:
```
HvResult res = HvApi.hv_vcpu_get_...(..., out value);
if (res == HvResult.BadArgument)
{
_fallbackCount++;
LogHvWarning("..."); // rate-limited
return cachedValue; // safe fallback
}
res.ThrowOnError(); // only throw on real errors
return cachedValue = value; // success path + cache update
```
* **AggressiveMode Global Option** (NOT IMPLEMENTED IN THIS PR, BUT A FUTURE PR WILL ADD THIS FULLY)
* Added:
`public static bool AggressiveMode { get; set; } = false;`
* When true/enabled, all warning logs in LogHvWarning() are suppressed.
* **Reasoning:** Diagnostic logging has overhead. In the future, users will be able to disable/enable this when required. This will also be useful for any dev working on macOS Hypevisor (aka - me).
* **Hot-Path Optimizations**
* Changed index checks from `if (index < 0 || index > 30)` to `if ((uint)index > 30)` in `GetX`, `SetX`, `GetV`, and `SetV`.
* Separated cache initialization into a dedicated `InitializeCacheDefaults()` method called by constructor and `Reset()`.
* **Reasoning:** `GetX` is called millions of times per second. Reducing branches improves CPU branch prediction and overall throughput. The clean initialization pattern ensures reliable behavior when vCPUs are reused from the pool.
* **Improved Logging, Diagnostics & Monitoring**
* Rate-limited warnings with a cooldown timer (`WarningCooldownTicks = 1_000_000_000 ≈ 1 second`).
* Clear, descriptive messages (e.g. "PAC failure on SP_EL0", "PAC failure on X15").
* Added `public long GetFallbackCount()` for runtime monitoring.
* **Reasoning:** We need visibility into fallback frequency during development and testing, but we must not spam the log for end users during normal gameplay.
_Quick side note:_
_The logging mentions "PAC failure" rather than HV `BadArgument`, and this is a shorthand I've added right now for myself.
Technically, we are catching HvResult.BadArgument. It's called "PAC failure" in logs because PAC is a frequent trigger, especially with games like TOTK that use heavy mods (AKA TOTK Optimiser)._
_Right now, I'm calling it a PAC failure, but will later rename to `BadArgument` as I progress further on a certain issue that's been sitting in our GitHub for some time._
_Back to the main stuff:_
* **GPU Synchronization & _Potential_ 0 FPS Mitigation (`HvExecutionContext.cs`) (PR FOLLOW UP IN THE FUTURE)**
* Added adaptive `TryGpuSync()` called periodically in the main `Execute()` loop (every 10–12 execution steps).
* Added strategic `Thread.Yield()` after `SvcAarch64` handling and in sync points.
* Tuned counters based on real gameplay testing (% 10 for light sync, % 6 for stronger sync).
* **Reasoning:** Hypervisor accelerates the guest CPU significantly. Without periodic yielding/flushing, MoltenVK's presentation queue can starve, causing permanent 0 FPS. The adaptive approach gives aims to fix this by performing a "resync". This has been tested and is showing positive progress, but, as mentioned earlier, will continue in a future PR.
These are all of the changes in this PR.
It's been quite a ride getting Hypervisor back up and running, but I loved every second of it.
If there is anything that is unclear in this PR, please let me know and I'll provide more details and update the description above.
Here is a small FAQ section to answer some more questions. These answers come from my own personal testing and behaviour that I observed.
### FAQ
**Q:** Why can some users play games with Hypervisor enabled and others can't on newer macOS versions?
**A:** This needs a bit of clarification.
You _can_ still run games with Hypervisor enabled on latest macOS - just not all games. Games such as Pokémon Legends: Z-A, The Legend of Zelda: Tears of the Kingdom, The Legend of Zelda: Breath of the Wild, can all _run_ and will _boot_ with Hypervisor enabled.
However, some other games, like Beyblade X Xone, Beyblade X Evobattle, Tales of Xillia Remastered, Tales of Berserk Remastered, etc, will _crash_ on boot with `BadArgument` error.
So it's not that Hypervisor works for one user and doesn't work for the other.
_It's that the game doesn't cause Hypervisor to crash._
Games differ in how aggressively they read registers early in boot and how much they rely on modded/JIT code. Games with heavy early supervisor calls and/or mods that manipulate pointers/PAC (pointer authentication errors) are far more likely to hit BadArgument.
(PAC is the reason why TOTK crashes with UltraCam when Hypervisor is disabled on macOS)
Lightweight or less modded titles often avoid the problematic code paths.
––––––
**Q:** How stable is this fix?
**A:** Updating macOS versions (Tahoe -> Tahoe) didn't break this implementation. Current data and testing all provide positive results (i.e. - no crash, hooray!)
––––––
**Q:** What about the `Denied` error?
**A:** This has more so to do with macOS permissions. Currently, Ryujinx is not being shipped in a DMG (which, by Apple conventions, is a more proper way of handling macOS app installations outside of the App Store). There is a theory that packaging Ryujinx into a DMG file will alleviate any permission issues that we currently have, and may help fix this type of error, so this PR may not even be needed (but we'll have to see how it goes).
That said, this error has not appeared in any of the more recent crash logs that I've analysed or acquired myself (from testing), leading to a possible hypothesis that the permission issues have been resolved, but the Hypervisor implementation broke on server builds due to an update to the framework. This is, however, unlikely, because, again, only builds after a particular version (i.e. - the last working version on a users system) broke.
However, DMG packaging is still on a table and something that we want to implement as soon as possible.
––––––
**Q:** So why do local builds run then?
**A:** A lot of different factors - build environment, folder location, lack of permissions associated with running .exe files, different ways of handling pure .exe files and .app bundles, notarisation, and more. There was no "scientific testing" done on each of the factors and seeing which one affects what exactly. Pushing this PR and implementing the DMG will tell.
Reviewed-on: https://git.ryujinx.app/projects/Ryubing/pulls/89
1570 lines
48 KiB
C#
1570 lines
48 KiB
C#
using ARMeilleure.State;
|
|
using Ryujinx.Common.Logging;
|
|
using Ryujinx.Cpu;
|
|
using Ryujinx.HLE.HOS.Kernel.Common;
|
|
using Ryujinx.HLE.HOS.Kernel.Process;
|
|
using Ryujinx.HLE.HOS.Kernel.SupervisorCall;
|
|
using Ryujinx.Horizon.Common;
|
|
using Ryujinx.Memory;
|
|
using System;
|
|
using System.Collections.Generic;
|
|
using System.Numerics;
|
|
using System.Text;
|
|
using System.Threading;
|
|
|
|
namespace Ryujinx.HLE.HOS.Kernel.Threading
|
|
{
|
|
class KThread : KSynchronizationObject, IKFutureSchedulerObject
|
|
{
|
|
private const int TlsUserDisableCountOffset = 0x100;
|
|
private const int TlsUserInterruptFlagOffset = 0x102;
|
|
|
|
// Tls -> ThreadType
|
|
private const int TlsThreadTypeOffsetAArch64 = 0x1F8;
|
|
private const int TlsThreadTypeOffsetAArch32 = 0x1FC;
|
|
|
|
// Tls -> ThreadType -> Version
|
|
private const int TlsThreadTypeVersionOffsetAArch64 = 0x46;
|
|
private const int TlsThreadTypeVersionOffsetAArch32 = 0x26;
|
|
|
|
// Tls -> ThreadType (Version 0) -> ThreadNamePointer
|
|
private const int TlsThreadTypeVersion0ThreadNamePointerOffsetAArch64 = 0x1A8;
|
|
private const int TlsThreadTypeVersion0ThreadNamePointerOffsetAArch32 = 0xE8;
|
|
|
|
// Tls -> ThreadType (Version 1) -> ThreadNamePointer
|
|
private const int TlsThreadTypeThreadNamePointerOffsetAArch64 = 0x1A0;
|
|
private const int TlsThreadTypeThreadNamePointerOffsetAArch32 = 0xE4;
|
|
|
|
|
|
public const int MaxWaitSyncObjects = 64;
|
|
|
|
private ManualResetEventSlim _schedulerWaitEvent;
|
|
|
|
public ManualResetEventSlim SchedulerWaitEvent => _schedulerWaitEvent;
|
|
|
|
public Thread HostThread { get; private set; }
|
|
|
|
public IExecutionContext Context { get; private set; }
|
|
|
|
public KThreadContext ThreadContext { get; private set; }
|
|
|
|
public int DynamicPriority { get; private set; }
|
|
public ulong AffinityMask { get; private set; }
|
|
|
|
public ulong ThreadUid { get; private set; }
|
|
|
|
public bool IsThreadNamed { get; set; }
|
|
|
|
private long _totalTimeRunning;
|
|
|
|
public long TotalTimeRunning => _totalTimeRunning;
|
|
|
|
public KSynchronizationObject SignaledObj { get; set; }
|
|
|
|
private ulong _entrypoint;
|
|
private ThreadStart _customThreadStart;
|
|
private bool _forcedUnschedulable;
|
|
|
|
public bool IsSchedulable => _customThreadStart == null && !_forcedUnschedulable;
|
|
|
|
public ulong MutexAddress { get; set; }
|
|
public int KernelWaitersCount { get; private set; }
|
|
|
|
public KProcess Owner { get; private set; }
|
|
|
|
private ulong _tlsAddress;
|
|
|
|
public ulong TlsAddress => _tlsAddress;
|
|
|
|
public KSynchronizationObject[] WaitSyncObjects { get; }
|
|
public int[] WaitSyncHandles { get; }
|
|
|
|
public long LastScheduledTime { get; set; }
|
|
|
|
public readonly LinkedListNode<KThread>[] SiblingsPerCore;
|
|
|
|
public LinkedList<KThread> Withholder { get; set; }
|
|
public readonly LinkedListNode<KThread> WithholderNode;
|
|
|
|
public readonly LinkedListNode<KThread> ProcessListNode;
|
|
|
|
private readonly LinkedList<KThread> _mutexWaiters;
|
|
private readonly LinkedListNode<KThread> _mutexWaiterNode;
|
|
|
|
private readonly LinkedList<KThread> _pinnedWaiters;
|
|
private readonly LinkedListNode<KThread> _pinnedWaiterNode;
|
|
|
|
public KThread MutexOwner { get; private set; }
|
|
|
|
public int ThreadHandleForUserMutex { get; set; }
|
|
|
|
private ThreadSchedState _forcePauseFlags;
|
|
private ThreadSchedState _forcePausePermissionFlags;
|
|
|
|
public Result ObjSyncResult { get; set; }
|
|
|
|
public int BasePriority { get; set; }
|
|
public int PreferredCore { get; set; }
|
|
|
|
public int CurrentCore { get; set; }
|
|
public int ActiveCore { get; set; }
|
|
|
|
public bool IsPinned { get; private set; }
|
|
|
|
private ulong _originalAffinityMask;
|
|
private int _originalPreferredCore;
|
|
private int _originalBasePriority;
|
|
private int _coreMigrationDisableCount;
|
|
|
|
public ThreadSchedState SchedFlags { get; private set; }
|
|
|
|
private int _shallBeTerminated;
|
|
|
|
private bool ShallBeTerminated => _shallBeTerminated != 0;
|
|
|
|
public bool TerminationRequested => ShallBeTerminated || SchedFlags == ThreadSchedState.TerminationPending;
|
|
|
|
public bool SyncCancelled { get; set; }
|
|
public bool WaitingSync { get; set; }
|
|
|
|
private int _hasExited;
|
|
private bool _hasBeenInitialized;
|
|
private bool _hasBeenReleased;
|
|
|
|
public bool WaitingInArbitration { get; set; }
|
|
|
|
private readonly Lock _activityOperationLock = new();
|
|
|
|
internal readonly ManualResetEventSlim DebugHalt = new(false);
|
|
|
|
public KThread(KernelContext context) : base(context)
|
|
{
|
|
WaitSyncObjects = new KSynchronizationObject[MaxWaitSyncObjects];
|
|
WaitSyncHandles = new int[MaxWaitSyncObjects];
|
|
|
|
SiblingsPerCore = new LinkedListNode<KThread>[KScheduler.CpuCoresCount];
|
|
|
|
for (int i = 0; i < SiblingsPerCore.Length; i++)
|
|
{
|
|
SiblingsPerCore[i] = new LinkedListNode<KThread>(this);
|
|
}
|
|
|
|
_mutexWaiters = [];
|
|
_pinnedWaiters = [];
|
|
|
|
WithholderNode = new LinkedListNode<KThread>(this);
|
|
ProcessListNode = new LinkedListNode<KThread>(this);
|
|
_mutexWaiterNode = new LinkedListNode<KThread>(this);
|
|
_pinnedWaiterNode = new LinkedListNode<KThread>(this);
|
|
}
|
|
|
|
public Result Initialize(
|
|
ulong entrypoint,
|
|
ulong argsPtr,
|
|
ulong stackTop,
|
|
int priority,
|
|
int cpuCore,
|
|
KProcess owner,
|
|
ThreadType type,
|
|
ThreadStart customThreadStart = null)
|
|
{
|
|
if ((uint)type > 3)
|
|
{
|
|
throw new ArgumentException($"Invalid thread type \"{type}\".");
|
|
}
|
|
|
|
PreferredCore = cpuCore;
|
|
AffinityMask |= 1UL << cpuCore;
|
|
|
|
SchedFlags = ThreadSchedState.None;
|
|
|
|
ActiveCore = cpuCore;
|
|
ObjSyncResult = KernelResult.ThreadNotStarted;
|
|
DynamicPriority = priority;
|
|
BasePriority = priority;
|
|
CurrentCore = cpuCore;
|
|
IsPinned = false;
|
|
|
|
_entrypoint = entrypoint;
|
|
_customThreadStart = customThreadStart;
|
|
|
|
if (type == ThreadType.User)
|
|
{
|
|
if (owner.AllocateThreadLocalStorage(out _tlsAddress) != Result.Success)
|
|
{
|
|
return KernelResult.OutOfMemory;
|
|
}
|
|
|
|
MemoryHelper.FillWithZeros(owner.CpuMemory, _tlsAddress, KTlsPageInfo.TlsEntrySize);
|
|
}
|
|
|
|
bool is64Bits;
|
|
|
|
if (owner != null)
|
|
{
|
|
Owner = owner;
|
|
|
|
owner.IncrementReferenceCount();
|
|
owner.IncrementThreadCount();
|
|
|
|
is64Bits = owner.Flags.HasFlag(ProcessCreationFlags.Is64Bit);
|
|
}
|
|
else
|
|
{
|
|
is64Bits = true;
|
|
}
|
|
|
|
HostThread = new Thread(ThreadStart) { Name = "HLE.KThread" };
|
|
|
|
Context = owner?.CreateExecutionContext() ?? new ProcessExecutionContext();
|
|
|
|
ThreadContext = new KThreadContext(Context);
|
|
|
|
Context.IsAarch32 = !is64Bits;
|
|
|
|
Context.SetX(0, argsPtr);
|
|
|
|
if (is64Bits)
|
|
{
|
|
Context.SetX(18, KSystemControl.GenerateRandom() | 1);
|
|
Context.SetX(31, stackTop);
|
|
}
|
|
else
|
|
{
|
|
Context.SetX(13, (uint)stackTop);
|
|
}
|
|
|
|
Context.TpidrroEl0 = (long)_tlsAddress;
|
|
Context.DebugPc = _entrypoint;
|
|
|
|
ThreadUid = KernelContext.NewThreadUid();
|
|
Context.ThreadUid = ThreadUid;
|
|
|
|
HostThread.Name = customThreadStart != null ? $"HLE.OsThread.{ThreadUid}" : $"HLE.GuestThread.{ThreadUid}";
|
|
|
|
_hasBeenInitialized = true;
|
|
|
|
_forcePausePermissionFlags = ThreadSchedState.ForcePauseMask;
|
|
|
|
if (owner != null)
|
|
{
|
|
owner.AddThread(this);
|
|
|
|
if (owner.IsPaused)
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if (TerminationRequested)
|
|
{
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
return Result.Success;
|
|
}
|
|
|
|
_forcePauseFlags |= ThreadSchedState.ProcessPauseFlag;
|
|
|
|
CombineForcePauseFlags();
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
}
|
|
}
|
|
|
|
return Result.Success;
|
|
}
|
|
|
|
public Result Start()
|
|
{
|
|
if (!KernelContext.KernelInitialized)
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if (!TerminationRequested)
|
|
{
|
|
_forcePauseFlags |= ThreadSchedState.KernelInitPauseFlag;
|
|
|
|
CombineForcePauseFlags();
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
}
|
|
|
|
Result result = KernelResult.ThreadTerminating;
|
|
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if (!ShallBeTerminated)
|
|
{
|
|
KThread currentThread = KernelStatic.GetCurrentThread();
|
|
|
|
while (SchedFlags != ThreadSchedState.TerminationPending && (currentThread == null || !currentThread.TerminationRequested))
|
|
{
|
|
if ((SchedFlags & ThreadSchedState.LowMask) != ThreadSchedState.None)
|
|
{
|
|
result = KernelResult.InvalidState;
|
|
break;
|
|
}
|
|
|
|
if (currentThread == null || currentThread._forcePauseFlags == ThreadSchedState.None)
|
|
{
|
|
if (Owner != null && _forcePauseFlags != ThreadSchedState.None)
|
|
{
|
|
CombineForcePauseFlags();
|
|
}
|
|
|
|
SetNewSchedFlags(ThreadSchedState.Running);
|
|
|
|
StartHostThread();
|
|
|
|
result = Result.Success;
|
|
break;
|
|
}
|
|
else
|
|
{
|
|
currentThread.CombineForcePauseFlags();
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if (currentThread.ShallBeTerminated)
|
|
{
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
return result;
|
|
}
|
|
|
|
public ThreadSchedState PrepareForTermination()
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
KThread currentThread = KernelStatic.GetCurrentThread();
|
|
|
|
if (Owner != null && currentThread != null && Owner.PinnedThreads[currentThread.CurrentCore] == this)
|
|
{
|
|
Owner.UnpinThread(this);
|
|
}
|
|
|
|
ThreadSchedState result;
|
|
|
|
if (Interlocked.Exchange(ref _shallBeTerminated, 1) == 0)
|
|
{
|
|
if ((SchedFlags & ThreadSchedState.LowMask) == ThreadSchedState.None)
|
|
{
|
|
SchedFlags = ThreadSchedState.TerminationPending;
|
|
}
|
|
else
|
|
{
|
|
if (_forcePauseFlags != ThreadSchedState.None)
|
|
{
|
|
_forcePauseFlags &= ~ThreadSchedState.ThreadPauseFlag;
|
|
|
|
ThreadSchedState oldSchedFlags = SchedFlags;
|
|
|
|
SchedFlags &= ThreadSchedState.LowMask;
|
|
|
|
AdjustScheduling(oldSchedFlags);
|
|
}
|
|
|
|
if (BasePriority >= 0x10)
|
|
{
|
|
SetPriority(0xF);
|
|
}
|
|
|
|
if ((SchedFlags & ThreadSchedState.LowMask) == ThreadSchedState.Running)
|
|
{
|
|
// TODO: GIC distributor stuffs (sgir changes ect)
|
|
Context.RequestInterrupt();
|
|
}
|
|
|
|
SignaledObj = null;
|
|
ObjSyncResult = KernelResult.ThreadTerminating;
|
|
|
|
ReleaseAndResume();
|
|
}
|
|
}
|
|
|
|
result = SchedFlags;
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
return result & ThreadSchedState.LowMask;
|
|
}
|
|
|
|
public void Terminate()
|
|
{
|
|
ThreadSchedState state = PrepareForTermination();
|
|
|
|
if (KernelStatic.GetCurrentThread() == this && state != ThreadSchedState.TerminationPending)
|
|
{
|
|
KernelContext.Synchronization.WaitFor(new KSynchronizationObject[] { this }, -1, out _);
|
|
}
|
|
}
|
|
|
|
public void HandlePostSyscall()
|
|
{
|
|
ThreadSchedState state;
|
|
|
|
do
|
|
{
|
|
if (TerminationRequested)
|
|
{
|
|
Exit();
|
|
|
|
// As the death of the thread is handled by the CPU emulator, we differ from the official kernel and return here.
|
|
break;
|
|
}
|
|
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if (TerminationRequested)
|
|
{
|
|
state = ThreadSchedState.TerminationPending;
|
|
}
|
|
else
|
|
{
|
|
if (_forcePauseFlags != ThreadSchedState.None)
|
|
{
|
|
CombineForcePauseFlags();
|
|
}
|
|
|
|
state = ThreadSchedState.Running;
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
} while (state == ThreadSchedState.TerminationPending);
|
|
}
|
|
|
|
public void Exit()
|
|
{
|
|
// TODO: Debug event.
|
|
|
|
if (Owner != null)
|
|
{
|
|
Owner.ResourceLimit?.Release(LimitableResource.Thread, 0, 1);
|
|
|
|
_hasBeenReleased = true;
|
|
}
|
|
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
_forcePauseFlags &= ~ThreadSchedState.ForcePauseMask;
|
|
_forcePausePermissionFlags = 0;
|
|
|
|
bool decRef = ExitImpl();
|
|
|
|
Context.StopRunning();
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
if (decRef)
|
|
{
|
|
DecrementReferenceCount();
|
|
}
|
|
}
|
|
|
|
private bool ExitImpl()
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
SetNewSchedFlags(ThreadSchedState.TerminationPending);
|
|
|
|
bool decRef = Interlocked.Exchange(ref _hasExited, 1) == 0;
|
|
|
|
Signal();
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
return decRef;
|
|
}
|
|
|
|
private int GetEffectiveRunningCore()
|
|
{
|
|
for (int coreNumber = 0; coreNumber < KScheduler.CpuCoresCount; coreNumber++)
|
|
{
|
|
if (KernelContext.Schedulers[coreNumber].CurrentThread == this)
|
|
{
|
|
return coreNumber;
|
|
}
|
|
}
|
|
|
|
return -1;
|
|
}
|
|
|
|
public Result Sleep(long timeout)
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if (TerminationRequested)
|
|
{
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
return KernelResult.ThreadTerminating;
|
|
}
|
|
|
|
SetNewSchedFlags(ThreadSchedState.Paused);
|
|
|
|
if (timeout > 0)
|
|
{
|
|
KernelContext.TimeManager.ScheduleFutureInvocation(this, timeout);
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
if (timeout > 0)
|
|
{
|
|
KernelContext.TimeManager.UnscheduleFutureInvocation(this);
|
|
}
|
|
|
|
return Result.Success;
|
|
}
|
|
|
|
public void SetPriority(int priority)
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if (IsPinned)
|
|
{
|
|
_originalBasePriority = priority;
|
|
}
|
|
else
|
|
{
|
|
BasePriority = priority;
|
|
}
|
|
|
|
UpdatePriorityInheritance();
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
}
|
|
|
|
public void Suspend(ThreadSchedState type)
|
|
{
|
|
_forcePauseFlags |= type;
|
|
|
|
CombineForcePauseFlags();
|
|
}
|
|
|
|
public void Resume(ThreadSchedState type)
|
|
{
|
|
ThreadSchedState oldForcePauseFlags = _forcePauseFlags;
|
|
|
|
_forcePauseFlags &= ~type;
|
|
|
|
if ((oldForcePauseFlags & ~type) == ThreadSchedState.None)
|
|
{
|
|
ThreadSchedState oldSchedFlags = SchedFlags;
|
|
|
|
SchedFlags &= ThreadSchedState.LowMask;
|
|
|
|
AdjustScheduling(oldSchedFlags);
|
|
}
|
|
}
|
|
|
|
public Result SetActivity(bool pause)
|
|
{
|
|
lock (_activityOperationLock)
|
|
{
|
|
Result result = Result.Success;
|
|
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
ThreadSchedState lowNibble = SchedFlags & ThreadSchedState.LowMask;
|
|
|
|
if (lowNibble is not ThreadSchedState.Paused and not ThreadSchedState.Running)
|
|
{
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
return KernelResult.InvalidState;
|
|
}
|
|
|
|
if (!TerminationRequested)
|
|
{
|
|
if (pause)
|
|
{
|
|
// Pause, the force pause flag should be clear (thread is NOT paused).
|
|
if ((_forcePauseFlags & ThreadSchedState.ThreadPauseFlag) == 0)
|
|
{
|
|
Suspend(ThreadSchedState.ThreadPauseFlag);
|
|
}
|
|
else
|
|
{
|
|
result = KernelResult.InvalidState;
|
|
}
|
|
}
|
|
else
|
|
{
|
|
// Unpause, the force pause flag should be set (thread is paused).
|
|
if ((_forcePauseFlags & ThreadSchedState.ThreadPauseFlag) != 0)
|
|
{
|
|
Resume(ThreadSchedState.ThreadPauseFlag);
|
|
}
|
|
else
|
|
{
|
|
result = KernelResult.InvalidState;
|
|
}
|
|
}
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
if (result == Result.Success && pause)
|
|
{
|
|
bool isThreadRunning = true;
|
|
|
|
while (isThreadRunning)
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if (TerminationRequested)
|
|
{
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
break;
|
|
}
|
|
|
|
isThreadRunning = false;
|
|
|
|
if (IsPinned)
|
|
{
|
|
KThread currentThread = KernelStatic.GetCurrentThread();
|
|
|
|
if (currentThread.TerminationRequested)
|
|
{
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
result = KernelResult.ThreadTerminating;
|
|
|
|
break;
|
|
}
|
|
|
|
_pinnedWaiters.AddLast(_pinnedWaiterNode);
|
|
|
|
currentThread.Reschedule(ThreadSchedState.Paused);
|
|
}
|
|
else
|
|
{
|
|
isThreadRunning = GetEffectiveRunningCore() >= 0;
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
}
|
|
}
|
|
|
|
return result;
|
|
}
|
|
}
|
|
|
|
public Result GetThreadContext3(out ThreadContext context)
|
|
{
|
|
context = default;
|
|
|
|
lock (_activityOperationLock)
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if ((_forcePauseFlags & ThreadSchedState.ThreadPauseFlag) == 0)
|
|
{
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
return KernelResult.InvalidState;
|
|
}
|
|
|
|
if (!TerminationRequested)
|
|
{
|
|
context = GetCurrentContext();
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
}
|
|
|
|
return Result.Success;
|
|
}
|
|
|
|
private static uint GetPsr(IExecutionContext context)
|
|
{
|
|
return context.Pstate & 0xFF0FFE20;
|
|
}
|
|
|
|
private const long ContextCacheDurationTicks = 800_000; // ~0.8ms cache lifetime
|
|
|
|
private ThreadContext _cachedContext;
|
|
private long _contextCacheTimestamp;
|
|
private bool _hasValidContextCache;
|
|
|
|
private ThreadContext GetCurrentContext()
|
|
{
|
|
var now = DateTime.UtcNow.Ticks;
|
|
|
|
// Cache hit
|
|
if (_hasValidContextCache && (now - _contextCacheTimestamp) < ContextCacheDurationTicks)
|
|
{
|
|
return _cachedContext;
|
|
}
|
|
|
|
const int MaxRegistersAArch32 = 15;
|
|
const int MaxFpuRegistersAArch32 = 16;
|
|
|
|
ThreadContext context = new();
|
|
Span<ulong> registersSpan = context.Registers.AsSpan();
|
|
Span<V128> fpuRegistersSpan = context.FpuRegisters.AsSpan();
|
|
|
|
if (Owner.Flags.HasFlag(ProcessCreationFlags.Is64Bit))
|
|
{
|
|
for (int i = 0; i < registersSpan.Length; i++)
|
|
{
|
|
registersSpan[i] = Context.GetX(i);
|
|
}
|
|
for (int i = 0; i < fpuRegistersSpan.Length; i++)
|
|
{
|
|
fpuRegistersSpan[i] = Context.GetV(i);
|
|
}
|
|
context.Fp = Context.GetX(29);
|
|
context.Lr = Context.GetX(30);
|
|
context.Sp = Context.GetX(31);
|
|
context.Pc = Context.Pc;
|
|
context.Pstate = GetPsr(Context);
|
|
context.Tpidr = (ulong)Context.TpidrroEl0;
|
|
}
|
|
else
|
|
{
|
|
for (int i = 0; i < MaxRegistersAArch32; i++)
|
|
{
|
|
registersSpan[i] = (uint)Context.GetX(i);
|
|
}
|
|
for (int i = 0; i < MaxFpuRegistersAArch32; i++)
|
|
{
|
|
fpuRegistersSpan[i] = Context.GetV(i);
|
|
}
|
|
context.Pc = (uint)Context.Pc;
|
|
context.Pstate = GetPsr(Context);
|
|
context.Tpidr = (uint)Context.TpidrroEl0;
|
|
}
|
|
|
|
context.Fpcr = (uint)Context.Fpcr;
|
|
context.Fpsr = (uint)Context.Fpsr;
|
|
|
|
// Update cache
|
|
_cachedContext = context;
|
|
_contextCacheTimestamp = now;
|
|
_hasValidContextCache = true;
|
|
|
|
return context;
|
|
}
|
|
|
|
public void CancelSynchronization()
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if ((SchedFlags & ThreadSchedState.LowMask) != ThreadSchedState.Paused || !WaitingSync)
|
|
{
|
|
SyncCancelled = true;
|
|
}
|
|
else if (Withholder != null)
|
|
{
|
|
Withholder.Remove(WithholderNode);
|
|
|
|
SetNewSchedFlags(ThreadSchedState.Running);
|
|
|
|
Withholder = null;
|
|
|
|
SyncCancelled = true;
|
|
}
|
|
else
|
|
{
|
|
SignaledObj = null;
|
|
ObjSyncResult = KernelResult.Cancelled;
|
|
|
|
SetNewSchedFlags(ThreadSchedState.Running);
|
|
|
|
SyncCancelled = false;
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
}
|
|
|
|
public Result SetCoreAndAffinityMask(int newCore, ulong newAffinityMask)
|
|
{
|
|
lock (_activityOperationLock)
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
bool isCoreMigrationDisabled = _coreMigrationDisableCount != 0;
|
|
|
|
// The value -3 is "do not change the preferred core".
|
|
if (newCore == -3)
|
|
{
|
|
newCore = isCoreMigrationDisabled ? _originalPreferredCore : PreferredCore;
|
|
|
|
if ((newAffinityMask & (1UL << newCore)) == 0)
|
|
{
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
return KernelResult.InvalidCombination;
|
|
}
|
|
}
|
|
|
|
if (isCoreMigrationDisabled)
|
|
{
|
|
_originalPreferredCore = newCore;
|
|
_originalAffinityMask = newAffinityMask;
|
|
}
|
|
else
|
|
{
|
|
ulong oldAffinityMask = AffinityMask;
|
|
|
|
PreferredCore = newCore;
|
|
AffinityMask = newAffinityMask;
|
|
|
|
if (oldAffinityMask != newAffinityMask)
|
|
{
|
|
int oldCore = ActiveCore;
|
|
|
|
if (oldCore >= 0 && ((AffinityMask >> oldCore) & 1) == 0)
|
|
{
|
|
if (PreferredCore < 0)
|
|
{
|
|
ActiveCore = sizeof(ulong) * 8 - 1 - BitOperations.LeadingZeroCount(AffinityMask);
|
|
}
|
|
else
|
|
{
|
|
ActiveCore = PreferredCore;
|
|
}
|
|
}
|
|
|
|
AdjustSchedulingForNewAffinity(oldAffinityMask, oldCore);
|
|
}
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
bool targetThreadPinned = true;
|
|
|
|
while (targetThreadPinned)
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if (TerminationRequested)
|
|
{
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
break;
|
|
}
|
|
|
|
targetThreadPinned = false;
|
|
|
|
int coreNumber = GetEffectiveRunningCore();
|
|
bool isPinnedThreadCurrentlyRunning = coreNumber >= 0;
|
|
|
|
if (isPinnedThreadCurrentlyRunning && ((1UL << coreNumber) & AffinityMask) == 0)
|
|
{
|
|
if (IsPinned)
|
|
{
|
|
KThread currentThread = KernelStatic.GetCurrentThread();
|
|
|
|
if (currentThread.TerminationRequested)
|
|
{
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
return KernelResult.ThreadTerminating;
|
|
}
|
|
|
|
_pinnedWaiters.AddLast(_pinnedWaiterNode);
|
|
|
|
currentThread.Reschedule(ThreadSchedState.Paused);
|
|
}
|
|
else
|
|
{
|
|
targetThreadPinned = true;
|
|
}
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
}
|
|
|
|
return Result.Success;
|
|
}
|
|
}
|
|
|
|
private void CombineForcePauseFlags()
|
|
{
|
|
ThreadSchedState oldFlags = SchedFlags;
|
|
ThreadSchedState lowNibble = SchedFlags & ThreadSchedState.LowMask;
|
|
|
|
SchedFlags = lowNibble | (_forcePauseFlags & _forcePausePermissionFlags);
|
|
|
|
AdjustScheduling(oldFlags);
|
|
}
|
|
|
|
private void SetNewSchedFlags(ThreadSchedState newFlags)
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
ThreadSchedState oldFlags = SchedFlags;
|
|
|
|
SchedFlags = (oldFlags & ThreadSchedState.HighMask) | newFlags;
|
|
|
|
if ((oldFlags & ThreadSchedState.LowMask) != newFlags)
|
|
{
|
|
AdjustScheduling(oldFlags);
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
}
|
|
|
|
public void ReleaseAndResume()
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if ((SchedFlags & ThreadSchedState.LowMask) == ThreadSchedState.Paused)
|
|
{
|
|
if (Withholder != null)
|
|
{
|
|
Withholder.Remove(WithholderNode);
|
|
|
|
SetNewSchedFlags(ThreadSchedState.Running);
|
|
|
|
Withholder = null;
|
|
}
|
|
else
|
|
{
|
|
SetNewSchedFlags(ThreadSchedState.Running);
|
|
}
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
}
|
|
|
|
public void Reschedule(ThreadSchedState newFlags)
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
ThreadSchedState oldFlags = SchedFlags;
|
|
|
|
SchedFlags = (oldFlags & ThreadSchedState.HighMask) |
|
|
(newFlags & ThreadSchedState.LowMask);
|
|
|
|
AdjustScheduling(oldFlags);
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
}
|
|
|
|
public void AddMutexWaiter(KThread requester)
|
|
{
|
|
AddToMutexWaitersList(requester);
|
|
|
|
requester.MutexOwner = this;
|
|
|
|
UpdatePriorityInheritance();
|
|
}
|
|
|
|
public void RemoveMutexWaiter(KThread thread)
|
|
{
|
|
if (thread._mutexWaiterNode?.List != null)
|
|
{
|
|
_mutexWaiters.Remove(thread._mutexWaiterNode);
|
|
}
|
|
|
|
thread.MutexOwner = null;
|
|
|
|
UpdatePriorityInheritance();
|
|
}
|
|
|
|
public KThread RelinquishMutex(ulong mutexAddress, out int count)
|
|
{
|
|
count = 0;
|
|
|
|
if (_mutexWaiters.First == null)
|
|
{
|
|
return null;
|
|
}
|
|
|
|
KThread newMutexOwner = null;
|
|
|
|
LinkedListNode<KThread> currentNode = _mutexWaiters.First;
|
|
|
|
do
|
|
{
|
|
// Skip all threads that are not waiting for this mutex.
|
|
while (currentNode != null && currentNode.Value.MutexAddress != mutexAddress)
|
|
{
|
|
currentNode = currentNode.Next;
|
|
}
|
|
|
|
if (currentNode == null)
|
|
{
|
|
break;
|
|
}
|
|
|
|
LinkedListNode<KThread> nextNode = currentNode.Next;
|
|
|
|
_mutexWaiters.Remove(currentNode);
|
|
|
|
currentNode.Value.MutexOwner = newMutexOwner;
|
|
|
|
if (newMutexOwner != null)
|
|
{
|
|
// New owner was already selected, re-insert on new owner list.
|
|
newMutexOwner.AddToMutexWaitersList(currentNode.Value);
|
|
}
|
|
else
|
|
{
|
|
// New owner not selected yet, use current thread.
|
|
newMutexOwner = currentNode.Value;
|
|
}
|
|
|
|
count++;
|
|
|
|
currentNode = nextNode;
|
|
}
|
|
while (currentNode != null);
|
|
|
|
if (newMutexOwner != null)
|
|
{
|
|
UpdatePriorityInheritance();
|
|
|
|
newMutexOwner.UpdatePriorityInheritance();
|
|
}
|
|
|
|
return newMutexOwner;
|
|
}
|
|
|
|
private void UpdatePriorityInheritance()
|
|
{
|
|
// If any of the threads waiting for the mutex has
|
|
// higher priority than the current thread, then
|
|
// the current thread inherits that priority.
|
|
int highestPriority = BasePriority;
|
|
|
|
if (_mutexWaiters.First != null)
|
|
{
|
|
int waitingDynamicPriority = _mutexWaiters.First.Value.DynamicPriority;
|
|
|
|
if (waitingDynamicPriority < highestPriority)
|
|
{
|
|
highestPriority = waitingDynamicPriority;
|
|
}
|
|
}
|
|
|
|
if (highestPriority != DynamicPriority)
|
|
{
|
|
int oldPriority = DynamicPriority;
|
|
|
|
DynamicPriority = highestPriority;
|
|
|
|
AdjustSchedulingForNewPriority(oldPriority);
|
|
|
|
if (MutexOwner != null)
|
|
{
|
|
// Remove and re-insert to ensure proper sorting based on new priority.
|
|
MutexOwner._mutexWaiters.Remove(_mutexWaiterNode);
|
|
|
|
MutexOwner.AddToMutexWaitersList(this);
|
|
|
|
MutexOwner.UpdatePriorityInheritance();
|
|
}
|
|
}
|
|
}
|
|
|
|
private void AddToMutexWaitersList(KThread thread)
|
|
{
|
|
LinkedListNode<KThread> nextPrio = _mutexWaiters.First;
|
|
|
|
int currentPriority = thread.DynamicPriority;
|
|
|
|
while (nextPrio != null && nextPrio.Value.DynamicPriority <= currentPriority)
|
|
{
|
|
nextPrio = nextPrio.Next;
|
|
}
|
|
|
|
if (nextPrio != null)
|
|
{
|
|
_mutexWaiters.AddBefore(nextPrio, thread._mutexWaiterNode);
|
|
}
|
|
else
|
|
{
|
|
_mutexWaiters.AddLast(thread._mutexWaiterNode);
|
|
}
|
|
}
|
|
|
|
private void AdjustScheduling(ThreadSchedState oldFlags)
|
|
{
|
|
if (oldFlags == SchedFlags)
|
|
{
|
|
return;
|
|
}
|
|
|
|
if (!IsSchedulable)
|
|
{
|
|
if (!_forcedUnschedulable)
|
|
{
|
|
// Ensure our thread is running and we have an event.
|
|
StartHostThread();
|
|
|
|
// If the thread is not schedulable, we want to just run or pause
|
|
// it directly as we don't care about priority or the core it is
|
|
// running on in this case.
|
|
|
|
if (SchedFlags == ThreadSchedState.Running)
|
|
{
|
|
_schedulerWaitEvent.Set();
|
|
}
|
|
else
|
|
{
|
|
_schedulerWaitEvent.Reset();
|
|
}
|
|
}
|
|
|
|
return;
|
|
}
|
|
|
|
if (oldFlags == ThreadSchedState.Running)
|
|
{
|
|
// Was running, now it's stopped.
|
|
if (ActiveCore >= 0)
|
|
{
|
|
KernelContext.PriorityQueue.Unschedule(DynamicPriority, ActiveCore, this);
|
|
}
|
|
|
|
for (int core = 0; core < KScheduler.CpuCoresCount; core++)
|
|
{
|
|
if (core != ActiveCore && ((AffinityMask >> core) & 1) != 0)
|
|
{
|
|
KernelContext.PriorityQueue.Unsuggest(DynamicPriority, core, this);
|
|
}
|
|
}
|
|
}
|
|
else if (SchedFlags == ThreadSchedState.Running)
|
|
{
|
|
// Was stopped, now it's running.
|
|
if (ActiveCore >= 0)
|
|
{
|
|
KernelContext.PriorityQueue.Schedule(DynamicPriority, ActiveCore, this);
|
|
}
|
|
|
|
for (int core = 0; core < KScheduler.CpuCoresCount; core++)
|
|
{
|
|
if (core != ActiveCore && ((AffinityMask >> core) & 1) != 0)
|
|
{
|
|
KernelContext.PriorityQueue.Suggest(DynamicPriority, core, this);
|
|
}
|
|
}
|
|
}
|
|
|
|
KernelContext.ThreadReselectionRequested = true;
|
|
}
|
|
|
|
private void AdjustSchedulingForNewPriority(int oldPriority)
|
|
{
|
|
if (SchedFlags != ThreadSchedState.Running || !IsSchedulable)
|
|
{
|
|
return;
|
|
}
|
|
|
|
// Remove thread from the old priority queues.
|
|
if (ActiveCore >= 0)
|
|
{
|
|
KernelContext.PriorityQueue.Unschedule(oldPriority, ActiveCore, this);
|
|
}
|
|
|
|
for (int core = 0; core < KScheduler.CpuCoresCount; core++)
|
|
{
|
|
if (core != ActiveCore && ((AffinityMask >> core) & 1) != 0)
|
|
{
|
|
KernelContext.PriorityQueue.Unsuggest(oldPriority, core, this);
|
|
}
|
|
}
|
|
|
|
// Add thread to the new priority queues.
|
|
KThread currentThread = KernelStatic.GetCurrentThread();
|
|
|
|
if (ActiveCore >= 0)
|
|
{
|
|
if (currentThread == this)
|
|
{
|
|
KernelContext.PriorityQueue.SchedulePrepend(DynamicPriority, ActiveCore, this);
|
|
}
|
|
else
|
|
{
|
|
KernelContext.PriorityQueue.Schedule(DynamicPriority, ActiveCore, this);
|
|
}
|
|
}
|
|
|
|
for (int core = 0; core < KScheduler.CpuCoresCount; core++)
|
|
{
|
|
if (core != ActiveCore && ((AffinityMask >> core) & 1) != 0)
|
|
{
|
|
KernelContext.PriorityQueue.Suggest(DynamicPriority, core, this);
|
|
}
|
|
}
|
|
|
|
KernelContext.ThreadReselectionRequested = true;
|
|
}
|
|
|
|
private void AdjustSchedulingForNewAffinity(ulong oldAffinityMask, int oldCore)
|
|
{
|
|
if (SchedFlags != ThreadSchedState.Running || DynamicPriority >= KScheduler.PrioritiesCount || !IsSchedulable)
|
|
{
|
|
return;
|
|
}
|
|
|
|
// Remove thread from the old priority queues.
|
|
for (int core = 0; core < KScheduler.CpuCoresCount; core++)
|
|
{
|
|
if (((oldAffinityMask >> core) & 1) != 0)
|
|
{
|
|
if (core == oldCore)
|
|
{
|
|
KernelContext.PriorityQueue.Unschedule(DynamicPriority, core, this);
|
|
}
|
|
else
|
|
{
|
|
KernelContext.PriorityQueue.Unsuggest(DynamicPriority, core, this);
|
|
}
|
|
}
|
|
}
|
|
|
|
// Add thread to the new priority queues.
|
|
for (int core = 0; core < KScheduler.CpuCoresCount; core++)
|
|
{
|
|
if (((AffinityMask >> core) & 1) != 0)
|
|
{
|
|
if (core == ActiveCore)
|
|
{
|
|
KernelContext.PriorityQueue.Schedule(DynamicPriority, core, this);
|
|
}
|
|
else
|
|
{
|
|
KernelContext.PriorityQueue.Suggest(DynamicPriority, core, this);
|
|
}
|
|
}
|
|
}
|
|
|
|
KernelContext.ThreadReselectionRequested = true;
|
|
}
|
|
|
|
public void SetEntryArguments(long argsPtr, int threadHandle)
|
|
{
|
|
Context.SetX(0, (ulong)argsPtr);
|
|
Context.SetX(1, (ulong)threadHandle);
|
|
}
|
|
|
|
public void TimeUp()
|
|
{
|
|
ReleaseAndResume();
|
|
}
|
|
|
|
public string GetGuestStackTrace()
|
|
{
|
|
return Owner.Debugger.GetGuestStackTrace(this);
|
|
}
|
|
|
|
public string GetGuestRegisterPrintout()
|
|
{
|
|
return Owner.Debugger.GetCpuRegisterPrintout(this);
|
|
}
|
|
|
|
public void PrintGuestStackTrace()
|
|
{
|
|
Logger.Info?.Print(LogClass.Cpu, $"Guest stack trace:\n{GetGuestStackTrace()}\n");
|
|
}
|
|
|
|
public void PrintGuestRegisterPrintout()
|
|
{
|
|
Logger.Info?.Print(LogClass.Cpu, $"Guest CPU registers:\n{GetGuestRegisterPrintout()}\n");
|
|
}
|
|
|
|
public void AddCpuTime(long ticks)
|
|
{
|
|
Interlocked.Add(ref _totalTimeRunning, ticks);
|
|
}
|
|
|
|
public void StartHostThread()
|
|
{
|
|
if (_schedulerWaitEvent == null)
|
|
{
|
|
ManualResetEventSlim schedulerWaitEvent = new(false);
|
|
|
|
if (Interlocked.Exchange(ref _schedulerWaitEvent, schedulerWaitEvent) == null)
|
|
{
|
|
HostThread.Start();
|
|
}
|
|
else
|
|
{
|
|
schedulerWaitEvent.Dispose();
|
|
}
|
|
}
|
|
}
|
|
|
|
private void ThreadStart()
|
|
{
|
|
_schedulerWaitEvent.Wait();
|
|
DebugHalt.Reset();
|
|
KernelStatic.SetKernelContext(KernelContext, this);
|
|
|
|
if (_customThreadStart != null)
|
|
{
|
|
_customThreadStart();
|
|
|
|
// Ensure that anything trying to join the HLE thread is unblocked.
|
|
Exit();
|
|
HandlePostSyscall();
|
|
}
|
|
else
|
|
{
|
|
Owner.Context.Execute(Context, _entrypoint);
|
|
}
|
|
|
|
Context.Dispose();
|
|
_schedulerWaitEvent.Dispose();
|
|
}
|
|
|
|
public void MakeUnschedulable()
|
|
{
|
|
_forcedUnschedulable = true;
|
|
}
|
|
|
|
public override bool IsSignaled()
|
|
{
|
|
return _hasExited != 0;
|
|
}
|
|
|
|
protected override void Destroy()
|
|
{
|
|
if (_hasBeenInitialized)
|
|
{
|
|
FreeResources();
|
|
|
|
bool released = Owner != null || _hasBeenReleased;
|
|
|
|
if (Owner != null)
|
|
{
|
|
Owner.ResourceLimit?.Release(LimitableResource.Thread, 1, released ? 0 : 1);
|
|
|
|
Owner.DecrementReferenceCount();
|
|
}
|
|
else
|
|
{
|
|
KernelContext.ResourceLimit.Release(LimitableResource.Thread, 1, released ? 0 : 1);
|
|
}
|
|
}
|
|
}
|
|
|
|
private void FreeResources()
|
|
{
|
|
Owner?.RemoveThread(this);
|
|
|
|
if (_tlsAddress != 0 && Owner.FreeThreadLocalStorage(_tlsAddress) != Result.Success)
|
|
{
|
|
throw new InvalidOperationException("Unexpected failure freeing thread local storage.");
|
|
}
|
|
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
// Wake up all threads that may be waiting for a mutex being held by this thread.
|
|
foreach (KThread thread in _mutexWaiters)
|
|
{
|
|
thread.MutexOwner = null;
|
|
thread._originalPreferredCore = 0;
|
|
thread.ObjSyncResult = KernelResult.InvalidState;
|
|
|
|
thread.ReleaseAndResume();
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
|
|
Owner?.DecrementThreadCountAndTerminateIfZero();
|
|
}
|
|
|
|
public void Pin()
|
|
{
|
|
IsPinned = true;
|
|
_coreMigrationDisableCount++;
|
|
|
|
int activeCore = ActiveCore;
|
|
|
|
_originalPreferredCore = PreferredCore;
|
|
_originalAffinityMask = AffinityMask;
|
|
|
|
ActiveCore = CurrentCore;
|
|
PreferredCore = CurrentCore;
|
|
AffinityMask = 1UL << CurrentCore;
|
|
|
|
if (activeCore != CurrentCore || _originalAffinityMask != AffinityMask)
|
|
{
|
|
AdjustSchedulingForNewAffinity(_originalAffinityMask, activeCore);
|
|
}
|
|
|
|
_originalBasePriority = BasePriority;
|
|
BasePriority = Math.Min(_originalBasePriority, BitOperations.TrailingZeroCount(Owner.Capabilities.AllowedThreadPriosMask) - 1);
|
|
UpdatePriorityInheritance();
|
|
|
|
// Disallows thread pausing
|
|
_forcePausePermissionFlags &= ~ThreadSchedState.ThreadPauseFlag;
|
|
CombineForcePauseFlags();
|
|
|
|
// TODO: Assign reduced SVC permissions
|
|
}
|
|
|
|
public void Unpin()
|
|
{
|
|
IsPinned = false;
|
|
_coreMigrationDisableCount--;
|
|
|
|
ulong affinityMask = AffinityMask;
|
|
int activeCore = ActiveCore;
|
|
|
|
PreferredCore = _originalPreferredCore;
|
|
AffinityMask = _originalAffinityMask;
|
|
|
|
if (AffinityMask != affinityMask)
|
|
{
|
|
if ((AffinityMask & 1UL << ActiveCore) != 0)
|
|
{
|
|
if (PreferredCore >= 0)
|
|
{
|
|
ActiveCore = PreferredCore;
|
|
}
|
|
else
|
|
{
|
|
ActiveCore = sizeof(ulong) * 8 - 1 - BitOperations.LeadingZeroCount((ulong)AffinityMask);
|
|
}
|
|
|
|
AdjustSchedulingForNewAffinity(affinityMask, activeCore);
|
|
}
|
|
}
|
|
|
|
BasePriority = _originalBasePriority;
|
|
UpdatePriorityInheritance();
|
|
|
|
if (!TerminationRequested)
|
|
{
|
|
// Allows thread pausing
|
|
_forcePausePermissionFlags |= ThreadSchedState.ThreadPauseFlag;
|
|
CombineForcePauseFlags();
|
|
|
|
// TODO: Restore SVC permissions
|
|
}
|
|
|
|
// Wake up waiters
|
|
foreach (KThread waiter in _pinnedWaiters)
|
|
{
|
|
waiter.ReleaseAndResume();
|
|
}
|
|
|
|
_pinnedWaiters.Clear();
|
|
}
|
|
|
|
public void SynchronizePreemptionState()
|
|
{
|
|
KernelContext.CriticalSection.Enter();
|
|
|
|
if (Owner != null && Owner.PinnedThreads[CurrentCore] == this)
|
|
{
|
|
ClearUserInterruptFlag();
|
|
|
|
Owner.UnpinThread(this);
|
|
}
|
|
|
|
KernelContext.CriticalSection.Leave();
|
|
}
|
|
|
|
public ushort GetUserDisableCount()
|
|
{
|
|
return Owner.CpuMemory.Read<ushort>(_tlsAddress + TlsUserDisableCountOffset);
|
|
}
|
|
|
|
public void SetUserInterruptFlag()
|
|
{
|
|
Owner.CpuMemory.Write<ushort>(_tlsAddress + TlsUserInterruptFlagOffset, 1);
|
|
}
|
|
|
|
public void ClearUserInterruptFlag()
|
|
{
|
|
Owner.CpuMemory.Write<ushort>(_tlsAddress + TlsUserInterruptFlagOffset, 0);
|
|
}
|
|
|
|
public string GetThreadName()
|
|
{
|
|
try
|
|
{
|
|
ulong threadNamePtr = 0;
|
|
if (Context.IsAarch32)
|
|
{
|
|
uint threadTypePtr32 = Owner.CpuMemory.Read<uint>(_tlsAddress + TlsThreadTypeOffsetAArch32);
|
|
if (threadTypePtr32 == 0)
|
|
{
|
|
return "";
|
|
}
|
|
|
|
ushort version = Owner.CpuMemory.Read<ushort>(threadTypePtr32 + TlsThreadTypeVersionOffsetAArch32);
|
|
switch (version)
|
|
{
|
|
case 0x0000:
|
|
case 0xFFFF:
|
|
threadNamePtr = Owner.CpuMemory.Read<uint>(threadTypePtr32 + TlsThreadTypeVersion0ThreadNamePointerOffsetAArch32);
|
|
break;
|
|
case 0x0001:
|
|
threadNamePtr = Owner.CpuMemory.Read<uint>(threadTypePtr32 + TlsThreadTypeThreadNamePointerOffsetAArch32);
|
|
break;
|
|
default:
|
|
Logger.Warning?.Print(LogClass.Kernel, $"Unknown ThreadType struct version: {version}");
|
|
break;
|
|
}
|
|
}
|
|
else
|
|
{
|
|
ulong threadTypePtr64 = Owner.CpuMemory.Read<ulong>(_tlsAddress + TlsThreadTypeOffsetAArch64);
|
|
if (threadTypePtr64 == 0)
|
|
{
|
|
return "";
|
|
}
|
|
|
|
ushort version = Owner.CpuMemory.Read<ushort>(threadTypePtr64 + TlsThreadTypeVersionOffsetAArch64);
|
|
switch (version)
|
|
{
|
|
case 0x0000:
|
|
case 0xFFFF:
|
|
threadNamePtr = Owner.CpuMemory.Read<ulong>(threadTypePtr64 + TlsThreadTypeVersion0ThreadNamePointerOffsetAArch64);
|
|
break;
|
|
case 0x0001:
|
|
threadNamePtr = Owner.CpuMemory.Read<ulong>(threadTypePtr64 + TlsThreadTypeThreadNamePointerOffsetAArch64);
|
|
break;
|
|
default:
|
|
Logger.Warning?.Print(LogClass.Kernel, $"Unknown ThreadType struct version: {version}");
|
|
break;
|
|
}
|
|
}
|
|
|
|
if (threadNamePtr == 0)
|
|
{
|
|
return "";
|
|
}
|
|
|
|
List<byte> nameBytes = new();
|
|
for (int i = 0; i < 0x20; i++)
|
|
{
|
|
byte b = Owner.CpuMemory.Read<byte>(threadNamePtr + (ulong)i);
|
|
if (b == 0)
|
|
{
|
|
break;
|
|
}
|
|
nameBytes.Add(b);
|
|
}
|
|
return Encoding.UTF8.GetString(nameBytes.ToArray());
|
|
|
|
} catch (InvalidMemoryRegionException)
|
|
{
|
|
Logger.Warning?.Print(LogClass.Kernel, "Failed to get thread name.");
|
|
return "";
|
|
} catch (Exception e) {
|
|
Logger.Error?.Print(LogClass.Kernel, $"Error getting thread name: {e.Message}");
|
|
return "";
|
|
}
|
|
}
|
|
}
|
|
}
|