Package: hippolyzer
HippolyzerHippolyzeris a revival of Linden Lab'sPyOGP librarytargeting modern Python 3, with a focus on debugging issues in Second Life-compatible
servers and clients. There is a secondary focus on mocking up new features without requiring a
modified server or client.Wherever reasonable, readability and testability are prioritized over performance.Almost all code from PyOGP has been either rewritten or replaced. Major changes from
upstream include making sure messages always correctly round-trip, and the addition
of a debugging proxy similar to ye olde WinGridProxy.It supports hot-reloaded addon scripts that can rewrite, inject or drop messages.
Also included are tools for working with SL-specific assets like the internal animation format,
and the internal mesh format.It's quick and easy to bash together a script that does something useful if you're familiar
with low-level SL details. See theLocal Animation addon example.SetupFrom SourcePython 3.10 or above isrequired. If you're unable to upgrade your system Python package due to
being on a stable distro, you can usepyenvto create
a self-contained Python install with the appropriate version.Create a clean Python 3 virtualenvwithpython -mvenv <virtualenv_dir>Activate the virtualenv by running the appropriate activation scriptUnder Linux this would be something likesource <virtualenv_dir>/bin/activateUnder Windows it's<virtualenv_dir>\Scripts\activate.batRunpip install hippolyzer, or runpip install -e .in a cloned repo to install an editable versionBinary Windows BuildsBinary Windows builds are available on theReleases page.
I don't extensively test these, building from source is recommended.ProxyA proxy is provided with both a CLI and Qt-based interface. The proxy application wraps a
custom SOCKS 5 UDP proxy, as well as an HTTP proxy based onmitmproxy.Multiple clients are supported at a time, and UDP messages may be injected in either
direction.Proxy SetupRun the proxy withhippolyzer-guiAddons can be loaded through theFile -> Manage Addonsmenu or on the command-line likehippolyzer-gui addon_examples/bezoscape.pyIf you want the command-line version, runhippolyzer-cliInstall the proxy's HTTPS certificate by going toFile -> Install HTTPS CertsYou can also install it withhippolyzer-cli --setup-ca <path to your viewer settings dir>.
On Linux that would be~/.firestorm_x64/if you're using Firestorm.Certificate validation can be disabled entirely through viewer debug settingNoVerifySSLCert,
but is not recommended.WindowsWindows viewers have broken SOCKS 5 proxy support. To work around that, you need to use a wrapper EXE that
can make the viewer to correctly talk to Hippolyzer. Follow the instructions onhttps://github.com/SaladDais/WinHippoAutoProxyto start the viewer and run it through Hippolyzer.The proxy shouldnotbe configured through the viewer's own preferences panel, it won't work correctly.OS X & LinuxSOCKS 5 works correctly on these platforms, so you can just configure it through thepreferences -> network -> proxy settingspanel:Start the viewer and configure it to use127.0.0.1:9061as a SOCKS proxy and127.0.0.1:9062as
an HTTP proxy. Youmustselect the option in the viewer to use the HTTP proxy for all HTTP
traffic, or logins will fail.Optionally, If you want to reduce HTTP proxy lag you can have asset requests bypass the HTTP proxy by setting
theno_proxyenv var appropriately. For ex.no_proxy="asset-cdn.glb.agni.lindenlab.com" ./firestorm.Log in!Or, if you're on Linux, you can instead useLinHippoAutoProxyto launch your viewer, which will configure everything for you. Note that connections from the in-viewer browser will
likelynotbe run through Hippolyzer when using LinHippoAutoProxy.FilteringBy default, the proxy's display filter is configured to ignore many high-frequency messages.
The filter field allows filtering on the presence of specific blocks or the values of
variables.For example, to find either chat messages mentioning "foo" or any message referencing125214in an ID field you could useChatFrom*.ChatData.Message~="foo" || *.*.*ID==125214. To find all
ObjectUpdates related to object ID125214you could do*ObjectUpdate*.ObjectData.*ID==125214 || *ObjectUpdate*.ObjectData.Data.*ID==125214to parse through both templated fields and fields inside the binaryDatafields for compressed and
terse object updates.Messages also have metadata attached that can be matched on. To match on all kinds of ObjectUpdates that were
related to the most recently selected object at the time the update was logged, you could do a filter likeMeta.ObjectUpdateIDs ~= Meta.SelectedLocalSimilarly, if you have multiple active sessions and are only interested in messages related to a specific
agent's session, you can do(Meta.AgentID == None || Meta.AgentID == "d929385f-41e3-4a34-a04e-f1fc39f24f12") && ....Vectors can also be compared. This will get any ObjectUpdate variant that occurs within a certain range:(*ObjectUpdate*.ObjectData.*Data.Position > (110, 50, 100) && *ObjectUpdate*.ObjectData.*Data.Position < (115, 55, 105))If you want to compare against an enum or a flag class in defined intemplates.py, you can just specify its name:ViewerEffect.Effect.Type == ViewerEffectType.EFFECT_BEAMLoggingDecoded messages are displayed in the log pane, clicking one will show the request and
response for HTTP messages, and a human-friendly form for UDP messages. Some messages and
fields havespecial packers definedthat will give a more human-readable form of enum or binary fields, with the original form beside or below it.For example, anAgentUpdatemessage may show up in the log pane like:OUT AgentUpdate
# 15136: <PacketFlags.ZEROCODED: 128>

[AgentData]
  AgentID = [[AGENT_ID]]
  SessionID = [[SESSION_ID]]
  BodyRotation = (0.0, 0.0, 0.06852579861879349, 0.9976493446715918)
  HeadRotation = (-0.0, -0.0, 0.05799926817417145, 0.998316625570896)
  # Many flag fields are unpacked as tuples with the original value next to them
  State =| ('EDITING',) #16
  CameraCenter = <120.69703674316406, 99.8336181640625, 59.547847747802734>
  CameraAtAxis = <0.9625586271286011, 0.11959066987037659, -0.243267223238945>
  CameraLeftAxis = <-0.12329451739788055, 0.992370069026947, 0.0>
  CameraUpAxis = <0.24141110479831696, 0.029993515461683273, 0.9699592590332031>
  Far = 88.0
  ControlFlags =| ('YAW_POS', 'NUDGE_AT_POS') #524544
  Flags =| ('HIDE_TITLE',) #1and anObjectImagefor setting a prim's texture may look likeOUT ObjectImage
# 3849: <PacketFlags.0: 0>

[AgentData]
  AgentID = [[AGENT_ID]]
  SessionID = [[SESSION_ID]]
[ObjectData]
  ObjectLocalID = 700966
  MediaURL = b''
  TextureEntry =| {'Textures': {None: '89556747-24cb-43ed-920b-47caed15465f'}, \
     'Color': {None: b'\xff\xff\xff\xff'}, \
     'ScalesS': {None: 1.0}, \
     'ScalesT': {None: 1.0}, \
     'OffsetsS': {None: 0}, \
     'OffsetsT': {None: 0}, \
     'Rotation': {None: 0}, \
     'BasicMaterials': {None: {'Bump': 0, 'FullBright': False, 'Shiny': 'MEDIUM'}}, \
     'MediaFlags': {None: {'WebPage': False, 'TexGen': 'DEFAULT', '_Unused': 0}}, \
     'Glow': {None: 0}, \
     'Materials': {None: '00000000-0000-0000-0000-000000000000'}}
  #TextureEntry = b'\x89UgG$\xcbC\xed\x92\x0bG\xca\xed\x15F_\x00\x00\x00\x00\x00\x00\x00\x00\x80?\x00\x00\x00\x80?\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'All unpackers also provide equivalent packers that work with the message builder.
The scripting interface uses the same packers as the logging interface, but uses a different
representation. Clicking the "Copy repr()" will give you a version of the message that you can paste into
an addon's script.Building MessagesThe proxy GUI includes a message builder similar to Alchemy's to allow building arbitrary messages, or
resending messages from the message log window. Both UDP and Caps messages may be sent.For example, here's a message that will drop a physical cube on your head:OUT ObjectAdd

[AgentData]
  # [[]] in a field value indicates a simple replacement
  # provided by the proxy
  AgentID = [[AGENT_ID]]
  SessionID = [[SESSION_ID]]
  GroupID = [[NULL_KEY]]
[ObjectData]
  # =| means the we should use the field's special packer mode
  # We treat PCode as an enum, so we'll convert from its string name to its int val
  PCode =| 'PRIMITIVE'
  Material = 3
  # With =| you may represent flags as a tuple of strings rather than an int
  # The only allowed flags in ObjectAdd are USE_PHYSICS (1) and CREATE_SELECTED (2)
  AddFlags =| ('USE_PHYSICS',)
  PathCurve = 16
  ProfileCurve = 1
  PathBegin = 0
  PathEnd = 0
  PathScaleX = 100
  PathScaleY = 100
  PathShearX = 0
  PathShearY = 0
  PathTwist = 0
  PathTwistBegin = 0
  PathRadiusOffset = 0
  PathTaperX = 0
  PathTaperY = 0
  PathRevolutions = 0
  PathSkew = 0
  ProfileBegin = 0
  ProfileEnd = 0
  ProfileHollow = 0
  BypassRaycast = 1
  # =$ indicates an eval()ed field, this will result in a vector 3m above the agent.
  RayStart =$ AGENT_POS + Vector3(0, 0, 3)
  # We can reference whatever we put in `RayStart` by accessing `block`
  RayEnd =$ block["RayStart"]
  RayTargetID = [[NULL_KEY]]
  RayEndIsIntersection = 0
  Scale = <0.5, 0.5, 0.5>
  Rotation = <0.0, 0.0, 0.0, 1.0>
  State = 0The repeat spinner at the bottom of the window lets you send a message multiple times.
anivariable is put into the eval context and can be used to vary messages across repeats.
With repeat set to two:OUT ChatFromViewer

[AgentData]
  AgentID = [[AGENT_ID]]
  SessionID = [[SESSION_ID]]
[ChatData]
  # Simple templated f-string
  Message =$ f'foo {i * 2}'
  Type =| 'NORMAL'
  Channel = 0will printUser: foo 0
User: foo 2
User: foo 4HTTP requests may be sent through the same window, with equivalent syntax for replacements
andeval()within the request body, if requested. As an example, sending a chat message
through theUntrustedSimulatorMessagecap would look like:POST [[UntrustedSimulatorMessage]] HTTP/1.1
Content-Type: application/llsd+xml
Accept: application/llsd+xml
X-Hippo-Directives: 1

<llsd>
<map>
   <key>message</key>
    <string>ChatFromViewer</string>
   <key>body</key>
    <map>
     <key>AgentData</key>
      <array>
        <map>
         <key>AgentID</key>
          <uuid><!HIPPOREPL[[AGENT_ID]]></uuid>
         <key>SessionID</key>
          <uuid><!HIPPOREPL[[SESSION_ID]]></uuid>
        </map>
      </array>
     <key>ChatData</key>
      <array>
        <map>
         <key>Channel</key>
          <integer>0</integer>
         <key>Message</key>
          <string>test <!HIPPOEVAL[[
            base64.b64encode(hex(1 + 1).encode("utf8"))
          ]]></string>
         <key>Type</key>
          <integer>1</integer>
        </map>
      </array>
    </map>
  </map>
</llsd>Addon commandsBy default, channel 524 is a special channel used for commands handled by addons'handle_commandhooks. For ex, an addon that supplies afoowith one string parameter
can be called by typing/524 foo somethingin chat./524 helpwill give you a list of all commands offered by currently loaded addons.Useful ExtensionsThese are quick and dirty, but should be viewer features. I'm not a viewer developer, so they're here.
If you are a viewer developer, please put them in a viewer.Local Animation - Allows loading and playing animations in LL's internal format from disk, replaying
when the animation changes on disk. Mostly useful for animators that want quick feedbackLocal Mesh - Allows specifying a target object to apply a mesh preview to. When a local mesh target
is specified, hitting the "calculate upload cost" button in the mesh uploader will instead
apply the mesh to the local mesh target. It works on attachments too. Useful for testing rigs before a
final, real upload.REPLA quick and dirty REPL is also included for when you want to do ad-hoc introspection of proxy state.
It can be launched at any time by typing/524 spawn_replin chat.The REPL is fully async aware and allows awaiting events without blocking:>>>fromhippolyzer.lib.client.object_managerimportObjectUpdateType>>>evt=awaitsession.objects.events.wait_for((ObjectUpdateType.UPDATE,),timeout=2.0)>>>evt.updated{'Position'}Potential ChangesAISv3 wrapper?Higher level wrappers for common things? I don't really need these, so only if people want to write them.Move things out oftemplates.py, right now most binary serialization stuff lives there
because it's more convenient for me to hot-reload.Ability to add menus?LicenseLGPLv3. If you have a good reason why, I might dual license.This packageincludes portions of the Second Life(TM) Viewer Artwork,
Copyright (C) 2008 Linden Research, Inc.  The viewer artwork is licensed under the Creative Commons
Attribution-Share Alike 3.0 License.ContributingEnsure that any patches are clean with no unnecessary whitespace or formatting changes, and that you
add new tests for any added functionality.PhilosophyWith a few notable exceptions, Hippolyzer focuses mainly on decomposition of data, and doesn't
provide many high-level abstractions for interpreting or manipulating that data. It's careful
to only do lossless transforms on data that are just prettier representations of the data sent
over the wire. Hippolyzer's goal is to help people understand how Second Life actually works,
automatically employing abstractions that hide how SL works is counter to that goal.For Client DevelopersThis section is mostly useful if you're developing a new SL-compatible client from scratch. Clients based
on LL's will work out of the box.Adding proxy support to a new clientHippolyzer's proxy application actually combines two proxies, aSOCKS 5UDP proxy and an HTTP proxy.To have your client's traffic proxied through Hippolyzer the general flow is:Open a TCP connection to Hippolyzer's SOCKS 5 proxy portThis should be done once per logical user session, as Hippolyzer assumes a 1:1 mapping of SOCKS TCP
connections to SL sessionsSend a UDP associate command without authenticationThe proxy will respond with a host / port pair that UDP messages may be sent throughAt this point you will no longer need to use the TCP connection, but it must be kept
alive until you want to break the UDP associationWhenever you send a UDP packet to a remote host, you'll need to instead send it to the host / port
from the UDP associate response. A SOCKS 5 header must be prepended to the data indicating the ultimate destination
of the packetAny received UDP packets will also have a SOCKS 5 header indicating the real source IP and addressWhen in doubt, checksocks_proxy.py,packets.pyand the SOCKS 5 RFC for more info on how to deal with SOCKS.https://github.com/SaladDais/WinHippoAutoProxy/blob/master/winhippoautoproxy/socks5udphooker.cppis a simple
example that wraps aroundrecvfrom()andsendto()and could be used as a starting point.All HTTP requests must be sent through the Hippolyzer's HTTP proxy port.You may not need to do any extra plumbing to get this to work if your chosen HTTP client
respects theHTTP_PROXYenvironment variable.All HTTPS connections will be encrypted with the proxy's TLS key. You'll need to either add it to whatever
CA bundle your client uses or disable certificate validation when a proxy is used.mitmproxy does its own certificate validation so disabling it in your client is OK.The proxy needs to use content sniffing to figure out which requests are login requests,
so make sure your request would passMITMProxyEventManager._is_login_request()Do I have to do all that?You might be able to automate some of it on Linux by usingLinHippoAutoProxy. If you're on Windows or MacOS the
above is your only option.Should I use this library to make an SL client in Python?Probably not. If you just want to write a client in Python, you should instead look at usinglibremetaversevia pythonnet.
I removed the client-related code inherited from PyOGP because libremetaverse's was simply better
for general use.https://github.com/CasperTech/node-metaverse/also looks like a good, modern wrapper if you
prefer TypeScript.There is, however, a very low-levelHippoClientclass provided for testing, but it's unlikely
to be what you want for writing a general-purpose bot.