XEP-0433: Extended Channel Search

The XMPP instant messaging ecosystem is a federated one. This leads to many different group chat service providers existing and interesting public group chats being spread out across them. In order to provide users with a way to find public group chats (henceforth called channels) of interest to them, there needs to be a way to execute a cross-domain search based on keywords.

The protocol in this document provides a general and extensible search for channels across different domains and service types (e.g. MUC vs. MIX). It provides meta-information right in the result set, which allows searching entities to skip additional Service Discovery (XEP-0030) [1] queries against the channels themselves.

The protocol is not only useful for cross-domain search, but also as an alternative to using a Service Discovery (XEP-0030) [1] disco#items request followed by many disco#info requests on a group chat service.

2. Requirements¶

3. Glossary¶

Channel: A public group chat hosted on a Group Chat Service. This can either be a Multi-User Chat (XEP-0045) [2] room, a Mediated Information eXchange (MIX) (XEP-0369) [3] channel or something else entirely.
Group Chat Service: An entity or deployment which offers multi-user chat relay, such as by Multi-User Chat (XEP-0045) [2] or Mediated Information eXchange (MIX) (XEP-0369) [3].
Search Service: An entity which offers the service described in this specification.
Searcher: An entity which requests information from the Search Service.

4. Use Cases¶

4.1 Announcing/discovering support¶

An entity annouces that it supports serving search queries by publishing the urn:xmpp:channel-search:0:search feature via Service Discovery (XEP-0030) [1]:

Example 1. XEP-0030 disco#info response¶

<iq from='search.service.example' to='client@user.example' id='id1' type='result'>
  <query xmlns='http://jabber.org/protocol/disco#info'>
    <!-- ... -->
    <feature var='urn:xmpp:channel-search:0:search'/>
    <!-- ... -->
  </query>
</iq>

4.2 Executing a keyword search¶

To execute a keyword search, the Searcher MAY first request the search form from the Search Service. Alternatively, the Searcher MAY use the form specified in this document with only the fields which must be implemented by the Search Service.

After obtaining the search form, the Searcher completes the form and sends it back to the Search Service. The Search Service replies with a Result Set Management (XEP-0059) [4] paginated list of results.

The search form is a form conforming to Field Standardization for Data Forms (XEP-0068) [5].

To request the search form, an entity sends an empty search element qualified by the urn:xmpp:channel-search:0:search namespace:

Example 2. Searcher requests form from the Search Service¶

<iq from='client@user.example' to='search.service.example' id='id2' type='get' xml:lang='en'>
  <search xmlns='urn:xmpp:channel-search:0:search'/>
</iq>

The Search Service replies with the form as in the following example:

Example 3. Search Service returns the search form¶

<iq from='search.service.example' to='client@user.example' id='id2' type='result' xml:lang='en'>
  <search xmlns='urn:xmpp:channel-search:0:search'>
    <x xmlns='jabber:x:data' type='form'>
      <field type='hidden' var='FORM_TYPE'>
        <value>urn:xmpp:channel-search:0:search-params</value>
      </field>
      <field type='text-single' var='q' label='Search for'/>
      <field type='boolean' var='all' label='Return all entries (ignore search terms)'/>
      <field type='boolean' var='sinname' label='Search in name'>
        <value>true</value>
      </field>
      <field type='boolean' var='sindescription' label='Search in description'>
        <value>true</value>
      </field>
      <field type='boolean' var='sinaddr' label='Search in address'>
        <value>true</value>
      </field>
      <field type='text-single' var='min_users' label='Minimum number of users'>
        <value>1</value>
      </field>
      <field var="types" type="list-multi" label="Service types">
        <value>xep-0045</value>
        <option label='XEP-0045 Multi User Chats'><value>xep-0045</value></option>
        <option label='XEP-0369 MIX channels'><value>xep-0369</value></option>
      </field>
      <field type='list-single' var='key' label='Sort results by'>
        <value>{urn:xmpp:channel-search:0:order}nusers</value>
        <option label='Number of online users'><value>{urn:xmpp:channel-search:0:order}nusers</value></option>
        <option label='Address'><value>{urn:xmpp:channel-search:0:order}address</value></option>
      </field>
    </x>
</search>
</iq>

Note: Not all of the fields shown above are mandatory to implement. See Search Form Fields for a list of fields and their implementation status.

4.2.2 Send a search request¶

To request the result list for a given search query, a Searcher submits a form with the urn:xmpp:channel-search:0:search-params FORM_TYPE. The Searcher MAY include a Result Set Management (XEP-0059) [4] <set/> element inside the <search/> element. In either case, the Search Service may reply with a RSM-paginated result and the Searcher MUST be able to process that.

If a Searcher composes a search request using a search form template obtained by the Search Service, it MAY omit all fields it does not know or where it does not change the value already supplied by the Search Service.

Example 4. Searcher submits a form to the Search Service¶

<iq from='client@user.example' to='search.service.example' id='id3' type='get' xml:lang='en'>
  <search xmlns='urn:xmpp:channel-search:0:search'>
    <set xmlns="http://jabber.org/protocol/rsm">
      <max>5</max>
    </set>
    <x xmlns="jabber:x:data" type="submit">
      <field type='hidden' var='FORM_TYPE'>
        <value>urn:xmpp:channel-search:0:search-params</value>
      </field>
      <field var="q" type="text-single" label="Search for">
        <value>xmpp.org</value>
      </field>
      <field var="key" type="list-single" label="Sort results by">
        <value>{urn:xmpp:channel-search:0:order}address</value>
        <option label='Number of online users'><value>{urn:xmpp:channel-search:0:order}nusers</value></option>
        <option label='Address'><value>{urn:xmpp:channel-search:0:order}address</value></option>
      </field>
    </x>
  </search>
</iq>

4.2.2.1 Success Case

The Search Service calculates the result, paginates it according to its own policy (possibly taking into account the pagination request from the client) and returns a single result page in the response IQ.

Example 5. Searcher submits a form to the Search Service¶

<iq from='search.service.example' to='client@user.example' id='id3' type='result' xml:lang='en'>
  <result xmlns='urn:xmpp:channel-search:0:search'>
    <item address='commteam@muc.xmpp.org'>
      <name>commteam</name>
      <nusers>10</nusers>
      <is-open/>
    </item>
    <!-- three more items -->
    <item address='operators@muc.xmpp.org'>
      <name>XMPP Service Operators</name>
      <description>Discussion venue for operators of federated XMPP services</description>
      <nusers>43</nusers>
      <is-open/>
    </item>
    <set xmlns='http://jabber.org/protocol/rsm'>
      <first>opaque-string-1</first>
      <last>opaque-string-2</last>
      <max>5</max>
    </set>
  </result>
</iq>

The result items are <item/> elements wrapped in a <result/> element qualified by the urn:xmpp:channel-search:0:search namespace. The schema, along with extension rules, is described in Result Item Format.

To obtain further results, the Searcher re-submits the identical form with an appropriate Result Set Management (XEP-0059) [4] pagination request, using the information provided by the Search Service in the result <set/> element.

4.2.2.2 Unsupported Sort Key

If the sort key requested by the Searcher is not supported by the Search Service, the Search Service MUST reply with <feature-not-implemented/> and the <invalid-sort-key> application defined condition and a modify type:

Example 6. Search Service replies with feature-not-implemented¶

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='modify'>
    <feature-not-implemented xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <invalid-sort-key xmlns='urn:xmpp:channel-search:0:error'/>
  </error>
</iq>

4.2.2.3 Invalid Search Terms

If the q field was supplied by the Searcher and the contents of the q field did not yield any term suitable for search, the Search Service MUST reply with an <bad-request/> error and the <invalid-search-terms/> application defined condition. The error type MUST be modify.

The server SHOULD include a human-readable description of the constraints for search terms which were not met in the <text/> element of the error.

Example 7. Search Service replies with the invalid-search-terms error¶

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='modify'>
    <bad-request xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'>
      Search terms must have at least three characters.
    </text>
    <invalid-search-terms xmlns='urn:xmpp:channel-search:0:error'/>
  </error>
</iq>

4.2.2.4 Rate Limiting

If the Search Service can not or does (by policy) not want to process the request due to excessive amounts of requests (either by the requesting entity, their domain or any other criteria), it MUST reply with an <resource-constraint/> error with type wait.

The application defined error condition <rate-limit/> MUST be included. This error condition has a RECOMMENDED attribute, retry-after, which provides the amount of seconds after which the Searcher MAY retry the request.

The Search Service MAY include a human-readable description of the rate limit and when to retry in the <text/> element.

Example 8. Search Service replies with a rate limit notification¶

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='wait'>
    <resource-constraint xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <rate-limit xmlns='urn:xmpp:channel-search:0:error' retry-after='30'/>
  </error>
</iq>

Note: See also the rate-limiting related business rules for Searcher entities.

4.2.2.5 Rejection of Full List Retrieval

If the Search Service can not or does (by policy) not want to allow a Searcher to retrieve the entire database of channels, it MUST reject queries which set the all field to true with an error as follows:

If the feature is generally disabled: <not-allowed/> with type cancel
If the feature is not offered to the Searcher based on its identity: <forbidden/> with type auth

In all cases, the application defined condition <full-set-retrieval-rejected/> MUST be included.

The Search Service MAY include a human-readable description of the restrictions around full-list retrieval.

For example, if the full set retrieval had been disabled service-wide by configuration, the Search Service would reply with the following error:

Example 9. Search Service replies with a full-set-retrieval-rejected error¶

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='cancel'>
    <not-allowed xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'>
      Retrieval of the full database is not allowed.
    </text>
    <full-set-retrieval-rejected xmlns='urn:xmpp:channel-search:0:error'/>
  </error>
</iq>

4.2.2.6 Conflicting Field Options

If the Searcher provides form fields which are conflicting, the Search Service MUST reply with a <bad-request/> error of type modify. In addition, the <conflicting-fields/> application specific condition MUST be included.

Conflicting field values are those which fundamentally cannot be used in the same query in such a way that the definition of their function is still adhered to. For example, q restricts the results by keywords, but all specifies that all entries are returned.

The Search Service SHOULD include a human-readable description of the conflicting fields, referencing to the label values of the involved fields.

The <conflicting-fields/> element MAY have one or more <var/> child elements which refer to var values of the submitted fields. At least one of the referenced fields must be changed in order for a follow-up query to succeed.`

For example, if the Searcher has set all to true and provided a query in q, the Search Service would reply with an error similar to the following:

Example 10. Search Service replies with a conflicting-fields error¶

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='cancel'>
    <not-allowed xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'>
      Cannot both return all results and search by keywords.
    </text>
    <conflicting-fields xmlns='urn:xmpp:channel-search:0:error'>
      <var>all</var>
      <var>q</var>
    </conflicting-fields>
  </error>
</iq>

4.2.2.7 No Search Conditions

If no field which would define a result set and which is understood by the Search Service is present, it MUST reply with a <bad-request/> error of type cancel.

In addition, the <no-search-conditions/> application defined condition MUST be included.

Example 11. Search Service replies with the no-search-conditions error¶

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='modify'>
    <bad-request xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <no-search-conditions xmlns='urn:xmpp:channel-search:0:error'/>
  </error>
</iq>

An example of this situation would be a form where neither q nor all are given.

5. Business Rules¶

6. Implementation Notes¶

6.1 Search Form Fields¶

The search form is extensible as per Field Standardization for Data Forms (XEP-0068) [5]. Implementations are free to add fields on both sides of the exchange, as long as they are properly namespaced using Clark Notation.

The following fields are specified by this document:

Table 1: Search Form Field Summary¶

var	type	Support level	Description
`q`	`text-single`	RECOMMENDED	Input for the keyword-based search. Conflicts with `all`.
`all`	`boolean`	OPTIONAL	Return all results, ignoring text search terms. This does not influence the restrictions imposed by the `types` field. Conflicts with `q`.
`sinaddress`	`boolean`	RECOMMENDED if `q` is supported	Control whether the keyword search searches in the address of the channel.
`sinname`	`boolean`	REQUIRED if `q` is supported	Control whether the keyword search searches in the name of the channel.
`sindescription`	`boolean`	REQUIRED if `q` is supported	Control whether the keyword search searches in the textual description of the channel.
`types`	`list-multi`	RECOMMENDED	Constrain the service types of channels to return. If not supported, the search MUST only cover Multi-User Chat (XEP-0045) [2] group chats.
`key`	`list-single`	REQUIRED	Select how the results are ordered.

The sort keys specified by this document are the following:

Table 2: Sort Key Values¶

Value	Description
`{urn:xmpp:channel-search:0:order}address`	Order the results by the address of the channel. This ordering mode guarantees that the Searcher gets a duplicate-free view without omissions when paginating.
`{urn:xmpp:channel-search:0:order}nusers`	Order the results descendingly by the number of users. This mode does not guarantee that all channels in the database are returned, nor does it guarantee that no duplicates occur across multiple pages.

6.1.1 Extensibility of the Search Form Fields¶

Search Service implementations may offer custom values for the key field, provided Clark Notation is used to namespace the values.

6.2 Result Item Format¶

The result items are <item/> elements qualified by the urn:xmpp:channel-search:0:search namespace.

Each <item/> element MUST have an address attribute whose value is a proper JID (as per either RFC 6122 [6] or RFC 7622 [7]). It identifies the channel uniquely.

The following child elements of <item/> are defined by this specification. They are all qualified by the same namespace as <item/> itself.

Table 3: Child elements of result items¶

Element name	Content model	Occurences	Description
`name`	text character data	1	The human-readable name of the channel.
`description`	text character data	1	The human-readable description of the channel.
`language`	text character data	1	A valid `xml:lang` code which indicates the primary language of the channel.
`nusers`	non-negative integer character data	1	Number of occupants
`service-type`	enumeration character data	1	The type of the service which hosts the channel. See below for values and semantics.
`is-open`	boolean character data	1	If set to true, it indicates that the channel can be joined without extra credentials.
`anonymity-mode`	enumeration character data	1	Anonymity level of participation. See below for values and semantics.

Notes:

Any child element may be omitted by a Search Service if the data is not available for any or all rooms.
The number of occupants may be stale by an undefined amount of time.
A service MAY return future versions of those elements alongside with past versions. Entities need to treat elements with the same name, but different namespace, as entirely different elements.

Table 4: Anonymity modes¶

Value	Description
`{urn:xmpp:channel-search:0:anonymity}none`	The bare JID of the account or the full JID of one or more devices of each occupant is visible to every other occupant.
`muc_semianonymous`	As specified in Multi-User Chat (XEP-0045) [2]

Table 5: Service types¶

Value	Description
`xep-0045`	A Multi-User Chat (XEP-0045) [2] service.
`xep-0369`	A Mediated Information eXchange (MIX) (XEP-0369) [3] service.

If a Search Service would return entries with the same address with different service types, it SHOULD prefer Mediated Information eXchange (MIX) (XEP-0369) [3] over Multi-User Chat (XEP-0045) [2]. Note that a Search Service MUST NOT return service types the client has not asked for.

6.2.1 Extensibility¶

Search Service implementations are free to add custom child elements to <item/> elements. Searcher implementations MUST be prepared to handle any unknown elements in <item/>, for example by ignoring them.

Additional values for the <anonymity-mode/> element may be specified by future extensions. If an implementation encounters an unknown value on this field, it is RECOMMENDED to either treat it as synonymous to {urn:xmpp:channel-search:0:anonymity}none or request the anonymity mode from the address using a protocol appropriate for the channel's service.

7. Security Considerations¶

When sending a search form with a q field, the Searcher transmits potentially sensitive information to a third party.

8. IANA Considerations¶

9. XMPP Registrar Considerations¶

This specification should probably create registries for the various fields it defines, as well as register a form type.

10. XML Schema¶

11. Design Considerations¶

Instead of rolling a custom protocol for the result items, Jabber Search (XEP-0055) [8] could have been used.

While the result format of Jabber Search (XEP-0055) [8] allows for some generality, it does so in a rather restricted way. It is limited by the data formats and types expressable in Data Forms (XEP-0004) [9]. Sturctured data, beyond lists of text and JIDs, can not be represented with Data Forms (XEP-0004) [9] at all. Machine-readable data would also have to be human-readable at the same time to provide a fallback view for human users. Interationalization of such human-readable data in field values is not possible with Data Forms (XEP-0004) [9].

The advantage of entities being able to process unknown fields in a degraded manner is, principally, still present in the current proposal (although with a different kind of degration).

Given the complexity of fully and correctly processing Data Forms (XEP-0004) [9] report data, the slim benefits did, in the eyes of the authors, not outweigh the costs.

12. Acknowledgements¶

The basis for this protocol was developed for the search.jabber.network public group chat search service. It has been cleaned up for publication as a Standards Track XEP by the author and modified to support more use-cases.

Appendices

Appendix A: Document Information¶

Appendix B: Author Information¶

Jonas Schäfer

Appendix C: Legal Notices¶

Copyright

Permissions

Permission is hereby granted, free of charge, to any person obtaining a copy of this specification (the "Specification"), to make use of the Specification without restriction, including without limitation the rights to implement the Specification in a software program, deploy the Specification in a network service, and copy, modify, merge, publish, translate, distribute, sublicense, or sell copies of the Specification, and to permit persons to whom the Specification is furnished to do so, subject to the condition that the foregoing copyright notice and this permission notice shall be included in all copies or substantial portions of the Specification. Unless separate permission is granted, modified works that are redistributed shall not contain misleading information regarding the authors, title, number, or publisher of the Specification, and shall not claim endorsement of the modified works by the authors, any organization or project to which the authors belong, or the XMPP Standards Foundation.

Disclaimer of Warranty

## NOTE WELL: This Specification is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. ##

Limitation of Liability

In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall the XMPP Standards Foundation or any author of this Specification be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising from, out of, or in connection with the Specification or the implementation, deployment, or other use of the Specification (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if the XMPP Standards Foundation or such author has been advised of the possibility of such damages.

IPR Conformance

This XMPP Extension Protocol has been contributed in full conformance with the XSF's Intellectual Property Rights Policy (a copy of which can be found at <https://xmpp.org/about/xsf/ipr-policy> or obtained by writing to XMPP Standards Foundation, P.O. Box 787, Parker, CO 80134 USA).

Visual Presentation

The HTML representation (you are looking at) is maintained by the XSF. It is based on the YAML CSS Framework, which is licensed under the terms of the CC-BY-SA 2.0 license.

Appendix D: Relation to XMPP¶

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 6120) and XMPP IM (RFC 6121) specifications contributed by the XMPP Standards Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this document has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Appendix E: Discussion Venue¶

The primary venue for discussion of XMPP Extension Protocols is the <standards@xmpp.org> discussion list.

Appendix F: Requirements Conformance¶

The following requirements keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".