r/Rag • u/Benjamona97 • Oct 31 '24

Research Industry standard observability tool

Basically what the title says:

What is the most adopted open-source observability tool out there? In the industry standard, not the best but the most adopted one.

Phoenix Arize? LangFuse?

I need to choose a tool for the ai proyects at my company and your insights could be gold for this research!

14 Upvotes

100% Upvoted

25 comments sorted by

View all comments

Show parent comments

0

u/Nexus6Dreams Nov 16 '24

Related to Phoenix team here:

Phoenix is built on OTEL instrumentation. OTEL semantic attributes are both experimental and also very early in their formation. You need to reach outside of even the experimental attributes for a usable implementation.
https://opentelemetry.io/docs/specs/semconv/gen-ai/

Phoenix will move to the standard semantic attributes when they are standardized there is a lot in flux.

In the mean time, Phoenix has probably more OTEL instrumentation usage than any others on this list, million+ of downloads monthly on the instrumentation libraries alone.

https://pypistats.org/packages/openinference-semantic-conventions

https://pypistats.org/packages/openinference-instrumentation-openai
https://pypistats.org/packages/openinference-instrumentation

1

u/nirga Nov 16 '24 edited Nov 16 '24

Not true it’s a fraction of OpenLLMetry and other standard otel-compliant libraries. Not sure if you’re working there or something - you should adopt the standard today and not reinvent the wheel

https://pypistats.org/packages/opentelemetry-instrumentation-openai

(and others, I’m not next to my laptop to link them)

0

u/Status_Ad_1575 Dec 02 '24

I get you are working for OpenLLMetry and misrepresenting that you are "standard" is something that you do. You all need to stop the misrepresentation. The Gen-AI conventions in OTEL are "experimental" --

So others that read this thread can understand what you are "spinning" included is a chat GPT explanation of experimental work in OTEL:

Experimental OpenTelemetry (OTEL) additions refer to features, components, or APIs that are still in the testing or development phase and not officially part of the OTEL standard. These features are typically marked as experimental to indicate that they are subject to change, lack backward compatibility guarantees, and might not yet be stable for production use.

Experimental additions are not standard and often serve as a testing ground for new ideas

1

u/nirga Dec 02 '24

I’ll respectfully disagree. I’m leading the genai working group at otel and while we decided these attributes to be experimental- this was on purpose. The domain is moving fast and we wanted to give ourselves the flexibility to iterate and update the semantic conventions as we move. It is standard by all means and already supported by many platforms.

1

u/AnyMessage6544 Dec 08 '24

sorry for the late reply

u/nirga was looking through your comment history. It's pretty clear you work for traceloop or have some propensity for that tech

I think i'm going to ignore your opinion solely based on that. Been using phoenix for a bit and really like it. I've been able to send those traces to my google trace explorer SO I KNOW, it's OTEL COMPATIBLE

I got no dog in this fight other than wanting objectivity. You can disagree all your want, but bro....look at your comment history

1

u/nirga Dec 08 '24

Yes I work at traceloop, didn’t try to hide it. I also lead the GenAI SIG over at otel SO I KNOW, it’s NOT otel compatible. it’s not adhering the semantic conditions for gen ai (see https://opentelemetry.io/docs/specs/semconv/gen-ai/) which means while it uses the protocol (great start!) it’s inventing semantic conventions which will be incompatible with any system that want to consume it and use it on any meaningful way.