Sign up Latest Topics
 
Categories > Questions, Answers, and Discussion > Measurement accuracy/precision?
 
 


Reply
  Author   Comment   Page 1 of 3      1   2   3   Next
jeffwink

Development Team Member
Registered:
Posts: 10
Posted  #1 
I have used another device that is measuring Hgb/SpO2 (Ember). See this page for a discussion of its accuracy and precision. http://www.cercacor.com/ember-hemoglobin-accuracy

I was unable to locate any similar description for the Moxy. I also scanned some recent research, but nothing jumped out.

So:

With regards to tHb, we are seeing pretty small changes. I have 5-1-5 assessments with the range of raw tHb values only varying .5 g/dL. The transitions from work to rest are often even smaller, e.g., 13.26 at cessation of work, peaking at 13.42 during 1 minute rest, and temporarily dipping to 13.14 within first 30secs of next work step.

Are these measurements noise or real differences?

This is not a criticism by any means, but I think it is important to understand this issue prior to putting a lot of weight behind any conclusions.

Obviously, this is an issue with ALL measurement/monitoring devices.

Accuracy and precision are of concern with the power meters out in the marketplace, too. A recent study highlights the shortcomings https://www.thieme-connect.de/products/ejournals/abstract/10.1055/s-0043-102945
bobbyjobling

Development Team Member
Registered:
Posts: 209
Posted  #2 
Interesting,  68% 1 Standard deviation normally industry's work to 95.xx%. So you need to multiply the Ember hemoglobin tolerance of 1 standard deviation by 2.

In this case, Ember accuracy is +/-2 g/dl and has a repeatability of 0.8 g/dl at 2 standard deviations = 95% confidence that the reading is within the stated tolerance. 

On some more critical measurements which require a 99.99% confidence you need to multiply 1 standard deviation tolerance value by 3.3



With MOXY I personally don't worry about absolute values, I just look at the trend during the assessment. I also don't compare previous assessment values. 




 
jeffwink

Development Team Member
Registered:
Posts: 10
Posted  #3 
Even if we just focus on the single-session readings, don't we need to know whether the individual readings are accurate/precise from moment to moment?
bobbyjobling

Development Team Member
Registered:
Posts: 209
Posted  #4 

If we have a true random value in the raw measurement, this can be reduced by taking an average of the values taken.
In Moxy software you can manually set the filter mode to 2 seconds average(default) or 0.5 seconds.
If you choose 0.5 you can see a reduction of repeatability as expected.
If you use excel you could further increase the filter time.

Accuracy and repeatability are two different methods to define the quality of a measurement equipment. If you don't have a known reference standard to measure the accuracy then what is important is the repeatability in the value given.

In my work, I use vacuum measurement equipment called Spinning Rotar Gauge it has an accuracy of 1% but if I set the filter mode to 2 seconds I'm not able to make any sense of the reading as it is very unstable. I need to change the filter to 15 seconds before I can use the value.




Roger

Moderator
Registered:
Posts: 245
Posted  #5 
The Moxy THb reading is not the same as the Ember.

The Moxy THb is related to total amount of hemoglobin and myoglobin in the optical path of the sensor.  Things like fat layer thickness, hydration, and blood volume in the muscle can all affect the reading.

The factors that affect the reading all stay pretty stable for a given sensor location for a short period of time except for the blood volume in the muscle.  This can change due to muscle tension or vasodilation.

So we just use the Moxy THb to identify changes in muscle blood volume.

I think the Ember is trying to measure the blood hemoglobin concentration which is how much hemoglobin there is per unit volume of blood.
runner

Development Team Member
Registered:
Posts: 56
Posted  #6 
Here's another recent article about Moxy and its accuracy:
https://www.ncbi.nlm.nih.gov/pubmed/28557670
ryinc

Development Team Member
Registered:
Posts: 359
Posted  #7 
Runner it seems to me they were using other physiologic parametera to infer Moxy reliabilty? If that is the correct understanding then that is surely a totally inappropriate approach?
bobbyjobling

Development Team Member
Registered:
Posts: 209
Posted  #8 
I need to read the full document but I agree with Ryan, looks like Moxy is compared with HR and VO2.

The ideal situation would be to create an artificial reference standard which simulates SmO2 and Thb. However, even if you had this reference standard, it is still important to recognise what can effect Moxy measurement, Roger mentioned some in the post above.










jeffwink

Development Team Member
Registered:
Posts: 10
Posted  #9 
Yes, the Ember is definitely measuring blood hemoglobin concentration. They are also able to offer accuracy/precision stats because they have compared their measurements to actual blood draws and lab analysis. 

It would appear that no such analog for THb is possible, i.e., no invasive test to compare results with. Perhaps there is for SmO2 but it also seems unlikely or at least extremely invasive/impractical.

I think the thing that is confusing is that the Moxy THb values "look" like g/dl numbers (in the correct range) even if it is actually an arbitrary unit.

The paper cited seems to say that at least at low-to-moderate intensity the Moxy SmO2 values correlate well with other measures of aerobic work (VO2 and HR). I suppose this is a starting point, but I guess I'm thinking we need something conceptually like:

The measurements are accurately describing the local conditions. While the actual numbers produced will vary between individuals and sensor locations, the scale of measurements resulting from a particular session accurately describes something meaningful about the O2 delivery and utilization. Additionally, these measurements are precise enough to describe systemic differences from session to session (readiness/recovery) and to identify meaningful changes or shifts in demand/supply (zoning). Further, interpretation of the Moxy measurements combined with external data taken in real-time, can identify limitations to performance and suggest training interventions.  

Thoughts?
sebo2000

Development Team Member
Registered:
Posts: 222
Posted  #10 
I found Moxy very consistent from session to session.

The only one instance where I found readings "strange" was last weekend.

Out and in 36km course. On the out, sun on the right side. Moxy readings from left deltoid increased substantially on the "in" part of the course, inconsistent with second Moxy on VL.
No usual drops of smo2 (indicating sun interference) just increased tHb which is strange, SmO2 looked fine.

To me it looked like sun has interrupted the tHb readings despite the fact that Moxy was covered with black plastic cover, but Smo2 was not affected. Very strange, I have never seen similar example.

Moxy does not show absolute values, but quite well displays SmO2 and blood volume change.
bobbyjobling

Development Team Member
Registered:
Posts: 209
Posted  #11 
You can also have indirect sun light scattering within the body effecting the reading.

You may have to shield a bigger area when the Moxy is on the deltoid.
ryinc

Development Team Member
Registered:
Posts: 359
Posted  #12 
Sebo i have seen something similar where the sun seemed to affect readings based on where it was relative to a moxy on deltoid. Then when i went into a shaded area it would be fine, so i think its quite likely what was happening to you.
CD

Development Team Member
Registered:
Posts: 6
Posted  #13 
New paper comparing Moxy to Portamon in the Journal of Biomedical Optics
https://www.ncbi.nlm.nih.gov/pubmed/29368457
bjrmd

Development Team Member
Registered:
Posts: 43
Posted  #14 
I was working on a "journal review" of that paper, just posted it out.
Jiri Dostal

Development Team Member
Registered:
Posts: 51
Posted  #15 
Ember device is based on Masimo technology. This is fundamentally different than Moxy. Massimo is well respected company, however their devices are not intended to be used in sports. Yes, their latest SpO2 sensor looks great, but those are ICU medical devices. We have tried Masimo devices in past in our exercise lab. Great for SpO2 readings. All other parameters invalid during hard exercises.

My personal view is that Ember will soon follow the path of BSX ....
Previous Topic | Next Topic
Print
Reply

Quick Navigation:

Easily create a Forum Website with Website Toolbox.

HTML hit counter - Quick-counter.net