Attaining randomness in FXs

edited February 2015 in Help request
Hi,

I have a quick question; say I have an FX that looks like this (taken from the tutorial):

[WobbleFX]
SlotList = Wobble

[Wobble]
Type = scale
StartTime = 0.0
EndTime = 1.0
StartValue = (1.0, 1.0, 1.0)
EndValue = (2.0, 2.0, 1.0) ~ (6.0, 6.0, 1.0)

When I try to apply this FX to multiple objects simultaneously, I can't observe the randomness in the "EndValue" attribute. I can see in <repo>/code/src/object/orxFX.c:573 that the FX is kept in a global table by value, and it's never recreated from the config while it's already being used by an object. Note that this doesn't even check the KeepInCache attribute of the FX section.

Is there any way to achieve randomness on each application of the FX?

Thanks

Comments

  • edited February 2015
    Hi!

    Yes, it's an unfortunate issue of caching the FXs. The rare few times I needed some random, I ended up using a few different versions (ie. Wobble1, Wobble2, ...) instead.

    The creation of FXs can be quite expensive (all proportions considered), however maybe there's a compromise that could be achieved here. Let's say, if I ever detect a random value in any of the slots of a FX, I could prevent it from ending up inside the cache table. I'd have to make sure it'd work correctly with hot reloading, but that shouldn't be too much of a problem.

    What do you think?
  • edited February 2015
    Hmm.. my opinion is divided between (1) your idea of detecting random values or (2) providing it as an optional attribute of the FX settings, such as a "MultipleInstance = true|false" attribute.

    One minus for (1) is that you have a nice abstraction about randomness in the config system, that is, if a value is X ~ Y, the result will be random each time we read it. Nice and simple. The problem is once you check whether a field is random or not and act accordingly, the mental model is not that simple anymore... The plus for (1), on the other hand, is that when all the instances of an FX is equivalent, caching is transparent and inconsequential, but once a field is random, then caching is not transparent anymore, and should therefore be avoided, justifying the fact that you check for randomness.

    The plus for (2) is that it's a simple and transparent solution.

    IMHO, the underlying problem here is that the semantics of an FX section is not well defined. Does the FX section describe a template or an instance? An "Object" section clearly defines an object template, also reflected in the fact that hot-loading the config file does not affect existing objects created from it. Then the FX section must be describing an instance; but what does it mean to have random values in an instance? Besides, viewing the FX section as describing an instance leaves us with no way of creating multiple instances, even programmatically, and I think this narrows the usefulness of the entire FX concept in Orx. It appears that if you need to do something slightly custom, you need to create an FX from scratch (orxFX_Create), read and assign its slots manually (no function, such as orxSlot_CreateFromConfig). Then you have to add this new FX to your objects through their orxFXPointers, which itself looks like an internal API.

    Maybe I'm deviating from the core of the issue, but I feel that the FX section should essentially behave like a template. If its construction and memory footprint are expensive, as you say, maybe we could cache it only when we know that the instances are equivalent (which sounds like (1) ). We could also make it optionally behave like a template (which sounds like (2) ).

    Hot-loading obviously makes all this more complicated, but why do you think existing orxOBJECTs don't need to be affected by a hot-load but orxFXs do need (template vs. instance issue again)?
  • edited February 2015
    Well, as you've mentioned, there are two categories of public-facing structures in orx, the templated ones and the instances.

    Object are manipulated all the time (and the API encourages that), whereas others are almost never modified and are cached as most as possible (timelines and tracks, sounds, FXs, shaders, ...). Some allow for some minor runtime variations, like the uniforms for shaders, where some as extremely static, such as the FXs. They were designed this way as it was their perceived use.

    I'm totally open to make it evolve though, that's why I'm happy to make the orxFX cacheable only when possible instead of the other way around. I personally prefer the first option as the second one, albeit more explicit, can also bring a new layer of user-made mistakes (discrepancies between the use of randoms and the explicit boolean attribute).

    That being said, there actually *is* an API to add slots from config: orxFX_AddSlotFromConfig(). :)
    One can thus modify the content of the slots in memory via the orxConfig API and then add those to a new FX, for example.

    I believe the config-based wrappers are always exposed when something gets data from config.
    The other way around isn't always true though, the only (big) example I can think of are the tracks, where it's only possible to create them from config with orxTimeLine_AddTrackFromConfig() and not directly in a programmatical fashion (although one can programmatically change the content of the config and then create a track).
  • edited February 2015
    iarwain wrote:
    I'm totally open to make it evolve though, that's why I'm happy to make the orxFX cacheable only when possible instead of the other way around. I personally prefer the first option as the second one, albeit more explicit, can also bring a new layer of user-made mistakes (discrepancies between the use of randoms and the explicit boolean attribute).
    Great, I'll probably come up with some suggestions about the API as I spend more time with FXs, but I think the ability to randomize certain elements would add great value to them. As for deciding on caching based on the availability of random values vs. explicit configuration; why not both :). You might want to disable caching for an FX even if it's deterministic. For example, you might want to read certain slots of an FX from config, but add some other slots programmatically. In such a scenario, it would be ugly to introduce some randomness just to switch off caching.

    That actually brings me to another issue that I'd like to raise. Currently, it's quite hard to access the FXs on an object. It's especially harder than it needs to be while you're adding a new FX to an object. The specific example I have in mind is when I needed to write a function called "addTerminalFX(Object * obj, string configSection)", whose purpose was to add an FX to an object and set the object's lifetime so that the object dies when the FX finishes. So, what I needed was to add the FX, and get the duration of the added FX. The API actually makes this quite difficult, as you need to retreive the orxFXPointer first through an undocumented orxStructure(...) call (is there any documentation about what structure holds what members?), iterate over the FXs inside the orxFXPointer and determine which one is the one you've just added. What I did instead was to create a brand new orxFX from config and read its lifetime instead, which, in addition to being a bit shady, would not work if the lifetime were random, and there was no caching. An alternative signature for orxObject_AddFX, which returns an orxFX * (returning orxNULL to signal failure) would make this task quite trivial.
    That being said, there actually *is* an API to add slots from config: orxFX_AddSlotFromConfig(). :)

    That's such a cryptic name :P. My bad :)
    One can thus modify the content of the slots in memory via the orxConfig API and then add those to a new FX, for example.

    That would be a global change though, right? So you'd have to constantly update those config values. Would you recommend using the config system for such temporary changes? I mean, say you have 10 objects, and you want to add some FX to each of them with some variation, so you update a config section for each of them, do you think the overhead here would be acceptable?
  • edited February 2015
    Great, I'll probably come up with some suggestions about the API as I spend more time with FXs, but I think the ability to randomize certain elements would add great value to them. As for deciding on caching based on the availability of random values vs. explicit configuration; why not both :). You might want to disable caching for an FX even if it's deterministic. For example, you might want to read certain slots of an FX from config, but add some other slots programmatically. In such a scenario, it would be ugly to introduce some randomness just to switch off caching.

    Mmh, why not creating a different FX in that case? Ie. instead of adding slots to a FX, one can add more separate FXs to an object.
    That actually brings me to another issue that I'd like to raise. Currently, it's quite hard to access the FXs on an object. It's especially harder than it needs to be while you're adding a new FX to an object. The specific example I have in mind is when I needed to write a function called "addTerminalFX(Object * obj, string configSection)", whose purpose was to add an FX to an object and set the object's lifetime so that the object dies when the FX finishes. So, what I needed was to add the FX, and get the duration of the added FX. The API actually makes this quite difficult, as you need to retreive the orxFXPointer first through an undocumented orxStructure(...) call (is there any documentation about what structure holds what members?), iterate over the FXs inside the orxFXPointer and determine which one is the one you've just added. What I did instead was to create a brand new orxFX from config and read its lifetime instead, which, in addition to being a bit shady, would not work if the lifetime were random, and there was no caching. An alternative signature for orxObject_AddFX, which returns an orxFX * (returning orxNULL to signal failure) would make this task quite trivial.

    I see. Usually I do it the other way around: I know when the object must get deleted and I create a customized FX accordingly.

    Would a orxObject_GetLastAddedFX(), similar to orxObject_GetLastAddedSound() help?

    Regarding:
    (is there any documentation about what structure holds what members?)

    I'm not sure what you mean by that.
    That would be a global change though, right? So you'd have to constantly update those config values. Would you recommend using the config system for such temporary changes? I mean, say you have 10 objects, and you want to add some FX to each of them with some variation, so you update a config section for each of them, do you think the overhead here would be acceptable?

    I was more thinking of it as creating a separate section inheriting from the original one and overriding whatever I need to override.
    But in any case, the overhead of modifying the FX, even in place, should be minimal.
  • edited February 2015
    iarwain wrote:
    Mmh, why not creating a different FX in that case? Ie. instead of adding slots to a FX, one can add more separate FXs to an object.

    You're right :)
    Would a orxObject_GetLastAddedFX(), similar to orxObject_GetLastAddedSound() help?

    Yes, that would be great.
    Regarding:
    (is there any documentation about what structure holds what members?)
    I'm not sure what you mean by that.

    I mean, the orxSTRUCTURE_GET_POINTER (and its helpers orxXXX(STRUCTURE)) calls are quite scary unless one digs deeply into the source code. Like when you need to access the FXs attached to an object, the API DOC has no indication of how one could do it. Trying really hard, you can notice that there's more you can access via _orxObject_GetStructure, by feeding it a "orxSTRUCTURE_ID _eStructureID", so, the next thing I do is to check the entries of the orxSTRUCTURE_ID enum. There, there's no indication of which of those I can access through an object instance. The tutorial also doesn't access the FXs through a pointer. Now I've discovered that in the actual source of the orxSTRUCTURE_ID enum there are comments indicating which ones you can access. Maybe it's me, but I feel that anything accessible through the orxStructure API is currently a bit cryptic.
    I was more thinking of it as creating a separate section inheriting from the original one and overriding whatever I need to override.
    But in any case, the overhead of modifying the FX, even in place, should be minimal.

    I see, maybe I should start viewing the whole config module as something more volatile. My intuition based on past experience tells me that I can modify config entries only as a last resort.

    I'm still a bit concerned about this though; Say I create a config section just to create an FX (or a timeline) for a specific object at a specific time. Then once I'm done with this temporary section, how do I make sure that it leaves no trace behind, so that I'm not leaking resources. I can't see any orxConfig_DeleteSection(). Even if I delete the section, how do I make sure that I'm not leaking the name string of the section? Will it stay cached somewhere? The entries inside the section are also an issue; when I trace what happens when you set a config value to orxConfig_InitValue, the value string gets stored in a global hash table in orxString_Store and I can't see anywhere in the code that removes anything from sstString.pstIDTable(the table that orxString_Store stores the strings in), so, any programmatically generated string I pass to the config module is basically a resource leak, isn't it?

    All of this is not a problem if one uses the config module to manage initial configuration, but (ab)using the config module at runtime raises all kinds of red flags in my intuition. Please tell me if the config module is designed for this and that I should adjust my intuition accordingly.
  • edited March 2015
    I mean, the orxSTRUCTURE_GET_POINTER (and its helpers orxXXX(STRUCTURE)) calls are quite scary unless one digs deeply into the source code. Like when you need to access the FXs attached to an object, the API DOC has no indication of how one could do it. Trying really hard, you can notice that there's more you can access via _orxObject_GetStructure, by feeding it a "orxSTRUCTURE_ID _eStructureID", so, the next thing I do is to check the entries of the orxSTRUCTURE_ID enum. There, there's no indication of which of those I can access through an object instance. The tutorial also doesn't access the FXs through a pointer. Now I've discovered that in the actual source of the orxSTRUCTURE_ID enum there are comments indicating which ones you can access. Maybe it's me, but I feel that anything accessible through the orxStructure API is currently a bit cryptic.

    It definitely can be improved. I can also add a showcase to one of the early tutorial. There are quite a few things that are not presented in the basics tutorials anymore: resources, console, profiler, hotloading, etc...
    I see, maybe I should start viewing the whole config module as something more volatile. My intuition based on past experience tells me that I can modify config entries only as a last resort.

    Actually it's supposed to be quite dynamic, the file inputs being only one of the ways to feed the system.
    I'm still a bit concerned about this though; Say I create a config section just to create an FX (or a timeline) for a specific object at a specific time. Then once I'm done with this temporary section, how do I make sure that it leaves no trace behind, so that I'm not leaking resources. I can't see any orxConfig_DeleteSection().

    It's called orxConfig_ClearSection(), along the lines of orxConfig_ClearValue() and orxConfig_Clear(). I reserve _Delete() when there are _Create() operations.
    Even if I delete the section, how do I make sure that I'm not leaking the name string of the section? Will it stay cached somewhere?

    By default everything will be cleared, including the name string, config-wise (not speaking about the StringID dictionary, more below). There's one way to protect that specific string to be freed: it's by calling orxConfig_ProtectSection(). It's just a counter-lock system so that other objects can just keep references on the section name instead of duplicating it locally. Now that there are StringIDs, it's less interesting but still available.
    The entries inside the section are also an issue; when I trace what happens when you set a config value to orxConfig_InitValue, the value string gets stored in a global hash table in orxString_Store and I can't see anywhere in the code that removes anything from sstString.pstIDTable(the table that orxString_Store stores the strings in), so, any programmatically generated string I pass to the config module is basically a resource leak, isn't it?

    It isn't exactly a leak, it's a StringID dictionary. It'll learn over time and will reuse entries when needed instead of making duplication. It also provide CRCs for quick comparisons.

    That being said, it's only used when a value is not a list (list are using duplicated strings as they're going to get processed further).

    I'm using dynamic values extensively in Little Cells, for example, where I end up with close to 100 000 entries in the dictionary but that amounts for ~2-3MB of data. If you get cases where you end up eating too much memory, I can go back and propose the two options as a config switch, I guess.
    All of this is not a problem if one uses the config module to manage initial configuration, but (ab)using the config module at runtime raises all kinds of red flags in my intuition. Please tell me if the config module is designed for this and that I should adjust my intuition accordingly.

    I mostly use it for dynamic content (I'm using it to easily communicate between tracks and code or to categorize objects on the fly without having to modify code, or even to add new tutorial properties or tracking data, again without having to touch the executable). I have about 1000 config calls per frame when the game runs, which amounts for ~0.6ms on my 6 year old computer. That's one of the biggest CPU eaters in Little Cells almost on par with FX updates (~0.65ms for 800 concurrent FXs).
    When starting the game I have about 3500 strings in the dictionary for about 99kb of RAM use.

    But again, all use cases might be different. For me Little Cells is the most hungry project I have to date but I'm sure other might come with games (ab)using the config system even more extensively and I'll then be happy to find a compromise.
  • edited March 2015
    Thanks a lot for this thorough clarification, I'll be much more tough with the config module in the future :) I still feel sour about strings getting cached indefinitely, but I'll follow your advice about not minding it unless it becomes a problem.

    P.S: Looking forward to the Steam release of Little Cells.
  • edited March 2015
    I'll be happy to reconsider the system if/when you encounter an issue when (ab)using the system. :)

    Thanks for Little Cells, it's coming along slowly, one Saturday night after the other. Integrating Steam is quite easy so far, especially compared to the nightmare that was integrating Ubi's UPlay.
  • edited June 2017
    A bit late (but better late than never, as they say :P): a new property for orxFX has been introduced: DoNotCache.
    When an FX is set with DoNotCache = true, every single instance of it will be entirely recreated from the config data, allowing for random values or modifications.

    It's here: https://github.com/orx/orx/commit/08093f1573c4749821f5a0e69f89ab3e043aab13
  • edited June 2017
    Yaay! :)
Sign In or Register to comment.