[27/59] libctf, next: introduce new class of easier-to-use iterators

Message ID 20200630233146.338613-28-nick.alcock@oracle.com
State New
Headers show
Series
  • Deduplicating CTF linker
Related show

Commit Message

H.J. Lu via Binutils June 30, 2020, 11:31 p.m.
The libctf machinery currently only provides one way to iterate over its
data structures: ctf_*_iter functions that take a callback and an arg
and repeatedly call it.

This *works*, but if you are doing a lot of iteration it is really quite
inconvenient: you have to package up your local variables into
structures over and over again and spawn lots of little functions even
if it would be clearer in a single run of code.  Look at ctf-string.c
for an extreme example of how unreadable this can get, with
three-line-long functions proliferating wildly.

The deduplicator takes this to the Nth level. It iterates over a whole
bunch of things: if we'd had to use _iter-class iterators for all of
them there would be twenty additional functions in the deduplicator
alone, for no other reason than that the iterator API requires it.

Let's do something better. strtok_r gives us half the design: generators
in a number of other languages give us the other half.

The *_next API allows you to iterate over CTF-like entities in a single
function using a normal while loop. e.g. here we are iterating over all
the types in a dict:

ctf_next_t *i = NULL;
int *hidden;
ctf_id_t id;

while ((id = ctf_type_next (fp, &i, &hidden, 1)) != CTF_ERR)
  {
    /* do something with 'hidden' and 'id' */
  }
if (ctf_errno (fp) != ECTF_NEXT_END)
    /* iteration error */

Here we are walking through the members of a struct with CTF ID
'struct_type':

ctf_next_t *i = NULL;
ssize_t offset;
const char *name;
ctf_id_t membtype;

while ((offset = ctf_member_next (fp, struct_type, &i, &name,
                                  &membtype)) >= 0
  {
    /* do something with offset, name, and membtype */
  }
if (ctf_errno (fp) != ECTF_NEXT_END)
    /* iteration error */

Like every other while loop, this means you have access to all the local
variables outside the loop while inside it, with no need to tiresomely
package things up in structures, move the body of the loop into a
separate function, etc, as you would with an iterator taking a callback.

ctf_*_next allocates 'i' for you on first entry (when it must be NULL),
and frees and NULLs it and returns a _next-dependent flag value when the
iteration is over: the fp errno is set to ECTF_NEXT_END when the
iteartion ends normally.  If you want to exit early, call
ctf_next_destroy on the iterator.  You can copy iterators using
ctf_next_copy, which copies their current iteration position so you can
remember loop positions and go back to them later (or ctf_next_destroy
them if you don't need them after all).

Each _next function returns an always-likely-to-be-useful property of
the thing being iterated over, and takes pointers to parameters for the
others: with very few exceptions all those parameters can be NULLs if
you're not interested in them, so e.g. you can iterate over only the
offsets of members of a structure this way:

while ((offset = ctf_member_next (fp, struct_id, &i, NULL, NULL)) >= 0)

If you pass an iterator in use by one iteration function to another one,
you get the new error ECTF_NEXT_WRONGFUN back; if you try to change
ctf_file_t in mid-iteration, you get ECTF_NEXT_WRONGFP back.

Internally the ctf_next_t remembers the iteration function in use,
various sizes and increments useful for almost all iterations, then
uses unions to overlap the actual entities being iterated over to keep
ctf_next_t size down.

Iterators available in the public API so far (all tested in actual use
in the deduplicator):

/* Iterate over the members of a STRUCT or UNION, returning each member's
   offset and optionally name and member type in turn.  On end-of-iteration,
   returns -1.  */
ssize_t
ctf_member_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
                 const char **name, ctf_id_t *membtype);

/* Iterate over the members of an enum TYPE, returning each enumerand's
   NAME or NULL at end of iteration or error, and optionally passing
   back the enumerand's integer VALue.  */
const char *
ctf_enum_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
              int *val);

/* Iterate over every type in the given CTF container (not including
   parents), optionally including non-user-visible types, returning
   each type ID and optionally the hidden flag in turn. Returns CTF_ERR
   on end of iteration or error.  */
ctf_id_t
ctf_type_next (ctf_file_t *fp, ctf_next_t **it, int *flag,
               int want_hidden);

/* Iterate over every variable in the given CTF container, in arbitrary
   order, returning the name and type of each variable in turn.  The
   NAME argument is not optional.  Returns CTF_ERR on end of iteration
   or error.  */
ctf_id_t
ctf_variable_next (ctf_file_t *fp, ctf_next_t **it, const char **name);

/* Iterate over all CTF files in an archive, returning each dict in turn as a
   ctf_file_t, and NULL on error or end of iteration.  It is the caller's
   responsibility to close it.  Parent dicts may be skipped.  Regardless of
   whether they are skipped or not, the caller must ctf_import the parent if
   need be.  */
ctf_file_t *
ctf_archive_next (const ctf_archive_t *wrapper, ctf_next_t **it,
                  const char **name, int skip_parent, int *errp);

ctf_label_next is prototyped but not implemented yet.

include/
	* ctf-api.h (ECTF_NEXT_END): New error.
	(ECTF_NEXT_WRONGFUN): Likewise.
	(ECTF_NEXT_WRONGFP): Likewise.
	(ECTF_NERR): Adjust.
	(ctf_next_t): New.
	(ctf_next_create): New prototype.
	(ctf_next_destroy): Likewise.
	(ctf_next_copy): Likewise.
	(ctf_member_next): Likewise.
	(ctf_enum_next): Likewise.
	(ctf_type_next): Likewise.
	(ctf_label_next): Likewise.
	(ctf_variable_next): Likewise.

libctf/
	* ctf-impl.h (ctf_next): New.
	(ctf_get_dict): New prototype.
	* ctf-lookup.c (ctf_get_dict): New, split out of...
	(ctf_lookup_by_id): ... here.
	* ctf-util.c (ctf_next_create): New.
	(ctf_next_destroy): New.
	(ctf_next_copy): New.
	* ctf-types.c (includes): Add <assert.h>.
	(ctf_member_next): New.
	(ctf_enum_next): New.
	(ctf_type_iter): Document the lack of iteration over parent
	types.
	(ctf_type_next): New.
	(ctf_variable_next): New.
	* ctf-archive.c (ctf_archive_next): New.
	* libctf.ver: Add new public functions.
---
 include/ctf-api.h    |  33 +++-
 libctf/ctf-archive.c | 106 +++++++++++++
 libctf/ctf-impl.h    |  32 ++++
 libctf/ctf-lookup.c  |  19 ++-
 libctf/ctf-types.c   | 352 ++++++++++++++++++++++++++++++++++++++++++-
 libctf/ctf-util.c    |  29 ++++
 libctf/libctf.ver    |   9 ++
 7 files changed, 570 insertions(+), 10 deletions(-)

-- 
2.27.0.247.g3dff7de930

Comments

H.J. Lu via Binutils July 22, 2020, 9:29 a.m. | #1
Don't post-modify here: GCC complains.
---
 libctf/ctf-archive.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libctf/ctf-archive.c b/libctf/ctf-archive.c
index 2e386a02dcf..4fbb246c67b 100644
--- a/libctf/ctf-archive.c
+++ b/libctf/ctf-archive.c
@@ -836,7 +836,8 @@ ctf_archive_next (const ctf_archive_t *wrapper, ctf_next_t **it, const char **na
 					 + sizeof (struct ctf_archive));
       nametbl = (((const char *) arc) + le64toh (arc->ctfa_names));
 
-      name_ = &nametbl[le64toh (modent[i->ctn_n++].name_offset)];
+      name_ = &nametbl[le64toh (modent[i->ctn_n].name_offset)];
+      i->ctn_n++;
     } while (skip_parent && strcmp (name_, _CTF_SECTION) == 0);
 
   if (name)
-- 
2.27.0.247.g3dff7de930

Patch

diff --git a/include/ctf-api.h b/include/ctf-api.h
index faa7b727fb8..47a1f732f0e 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -204,10 +204,13 @@  enum
    ECTF_DUMPSECTCHANGED, /* Section changed in middle of dump.  */
    ECTF_NOTYET,		/* Feature not yet implemented.  */
    ECTF_INTERNAL,	/* Internal error in link.  */
-   ECTF_NONREPRESENTABLE /* Type not representable in CTF.  */
+   ECTF_NONREPRESENTABLE, /* Type not representable in CTF.  */
+   ECTF_NEXT_END,	/* End of iteration.  */
+   ECTF_NEXT_WRONGFUN,	/* Wrong iteration function called.  */
+   ECTF_NEXT_WRONGFP	/* Iteration entity changed in mid-iterate.  */
   };
 
-#define ECTF_NERR (ECTF_NONREPRESENTABLE - ECTF_BASE + 1)	/* Count of CTF errors.  */
+#define ECTF_NERR (ECTF_NEXT_WRONGFP - ECTF_BASE + 1)	/* Count of CTF errors.  */
 
 /* The CTF data model is inferred to be the caller's data model or the data
    model of the given object, unless ctf_setmodel() is explicitly called.  */
@@ -227,8 +230,9 @@  enum
 #define	CTF_ADD_NONROOT	0	/* Type only visible in nested scope.  */
 #define	CTF_ADD_ROOT	1	/* Type visible at top-level scope.  */
 
-/* These typedefs are used to define the signature for callback functions
-   that can be used with the iteration and visit functions below.  */
+/* These typedefs are used to define the signature for callback functions that
+   can be used with the iteration and visit functions below.  There is also a
+   family of iteration functions that do not require callbacks.  */
 
 typedef int ctf_visit_f (const char *name, ctf_id_t type, unsigned long offset,
 			 int depth, void *arg);
@@ -248,6 +252,15 @@  typedef char *ctf_dump_decorate_f (ctf_sect_names_t sect,
 
 typedef struct ctf_dump_state ctf_dump_state_t;
 
+/* Iteration state for the _next() functions, and allocators/copiers/freers for
+   it.  (None of these are needed for the simple case of iterating to the end:
+   the _next() function allocate and free the iterators for you.)  */
+
+typedef struct ctf_next ctf_next_t;
+extern ctf_next_t *ctf_next_create (void);
+extern void ctf_next_destroy (ctf_next_t *);
+extern ctf_next_t *ctf_next_copy (ctf_next_t *);
+
 /* Opening.  These mostly return an abstraction over both CTF files and CTF
    archives: so they can be used to open both.  CTF files will appear to be an
    archive with one member named '.ctf'.  The low-level functions
@@ -353,13 +366,25 @@  extern int ctf_label_info (ctf_file_t *, const char *, ctf_lblinfo_t *);
 
 extern int ctf_member_count (ctf_file_t *, ctf_id_t);
 extern int ctf_member_iter (ctf_file_t *, ctf_id_t, ctf_member_f *, void *);
+extern ssize_t ctf_member_next (ctf_file_t *, ctf_id_t, ctf_next_t **,
+				const char **name, ctf_id_t *membtype);
 extern int ctf_enum_iter (ctf_file_t *, ctf_id_t, ctf_enum_f *, void *);
+extern const char *ctf_enum_next (ctf_file_t *, ctf_id_t, ctf_next_t **,
+				  int *);
 extern int ctf_type_iter (ctf_file_t *, ctf_type_f *, void *);
 extern int ctf_type_iter_all (ctf_file_t *, ctf_type_all_f *, void *);
+extern ctf_id_t ctf_type_next (ctf_file_t *, ctf_next_t **,
+			       int *flag, int want_hidden);
 extern int ctf_label_iter (ctf_file_t *, ctf_label_f *, void *);
+extern int ctf_label_next (ctf_file_t *, ctf_next_t **, const char **); /* TBD */
 extern int ctf_variable_iter (ctf_file_t *, ctf_variable_f *, void *);
+extern ctf_id_t ctf_variable_next (ctf_file_t *, ctf_next_t **,
+				   const char **);
 extern int ctf_archive_iter (const ctf_archive_t *, ctf_archive_member_f *,
 			     void *);
+extern ctf_file_t *ctf_archive_next (const ctf_archive_t *, ctf_next_t **,
+				     const char **, int skip_parent, int *errp);
+
 /* This function alone does not currently operate on CTF files masquerading
    as archives, and returns -EINVAL: the raw data is no longer available.  It is
    expected to be used only by archiving tools, in any case, which have no need
diff --git a/libctf/ctf-archive.c b/libctf/ctf-archive.c
index 1cfe40f6e3a..0d824356739 100644
--- a/libctf/ctf-archive.c
+++ b/libctf/ctf-archive.c
@@ -739,6 +739,112 @@  ctf_archive_iter (const ctf_archive_t *arc, ctf_archive_member_f *func,
   return func (arc->ctfi_file, _CTF_SECTION, data);
 }
 
+/* Iterate over all CTF files in an archive, returning each dict in turn as a
+   ctf_file_t, and NULL on error or end of iteration.  It is the caller's
+   responsibility to close it.  Parent dicts may be skipped.  Regardless of
+   whether they are skipped or not, the caller must ctf_import the parent if
+   need be.
+
+   We identify parents by name rather than by flag value: for now, with the
+   linker only emitting parents named _CTF_SECTION, this works well enough.  */
+
+ctf_file_t *
+ctf_archive_next (const ctf_archive_t *wrapper, ctf_next_t **it, const char **name,
+		  int skip_parent, int *errp)
+{
+  ctf_file_t *f;
+  ctf_next_t *i = *it;
+  struct ctf_archive *arc;
+  struct ctf_archive_modent *modent;
+  const char *nametbl;
+  const char *name_;
+
+  if (!i)
+    {
+      if ((i = ctf_next_create()) == NULL)
+	{
+	  if (errp)
+	    *errp = ENOMEM;
+	  return NULL;
+	}
+      i->cu.ctn_arc = wrapper;
+      i->ctn_iter_fun = (void (*) (void)) ctf_archive_next;
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_archive_next != i->ctn_iter_fun)
+    {
+      if (errp)
+	*errp = ECTF_NEXT_WRONGFUN;
+      return NULL;
+    }
+
+  if (wrapper != i->cu.ctn_arc)
+    {
+      if (errp)
+	*errp = ECTF_NEXT_WRONGFP;
+      return NULL;
+    }
+
+  /* Iteration is made a bit more complex by the need to handle ctf_file_t's
+     transparently wrapped in a single-member archive.  These are parents: if
+     skip_parent is on, they are skipped and the iterator terminates
+     immediately.  */
+
+  if (!wrapper->ctfi_is_archive && i->ctn_n == 0)
+    {
+      i->ctn_n++;
+      if (!skip_parent)
+	{
+	  wrapper->ctfi_file->ctf_refcnt++;
+	  return wrapper->ctfi_file;
+	}
+    }
+
+  arc = wrapper->ctfi_archive;
+
+  /* The loop keeps going when skip_parent is on as long as the member we find
+     is the parent (i.e. at most two iterations, but possibly an early return if
+     *all* we have is a parent).  */
+
+  const ctf_sect_t *symsect;
+  const ctf_sect_t *strsect;
+
+  do
+    {
+      if ((!wrapper->ctfi_is_archive) || (i->ctn_n >= le64toh (arc->ctfa_nfiles)))
+	{
+	  ctf_next_destroy (i);
+	  *it = NULL;
+	  if (errp)
+	    *errp = ECTF_NEXT_END;
+	  return NULL;
+	}
+
+      symsect = &wrapper->ctfi_symsect;
+      strsect = &wrapper->ctfi_strsect;
+
+      if (symsect->cts_name == NULL)
+	symsect = NULL;
+      if (strsect->cts_name == NULL)
+	strsect = NULL;
+
+      modent = (ctf_archive_modent_t *) ((char *) arc
+					 + sizeof (struct ctf_archive));
+      nametbl = (((const char *) arc) + le64toh (arc->ctfa_names));
+
+      name_ = &nametbl[le64toh (modent[i->ctn_n++].name_offset)];
+    } while (skip_parent && strcmp (name_, _CTF_SECTION) == 0);
+
+  if (name)
+    *name = name_;
+
+  f = ctf_arc_open_by_name_internal (arc, symsect, strsect,
+				     name_, errp);
+  f->ctf_archive = (ctf_archive_t *) wrapper;
+  return f;
+}
+
 #ifdef HAVE_MMAP
 /* Map the header in.  Only used on new, empty files.  */
 static void *arc_mmap_header (int fd, size_t headersz)
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 81a3c62dcfc..b18ba946524 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -326,6 +326,36 @@  struct ctf_archive_internal
   void (*ctfi_bfd_close) (struct ctf_archive_internal *);
 };
 
+/* Iterator state for the *_next() functions.  */
+
+struct ctf_next
+{
+  void (*ctn_iter_fun) (void);
+  ctf_id_t ctn_type;
+  ssize_t ctn_size;
+  ssize_t ctn_increment;
+  uint32_t ctn_n;
+  /* We can save space on this side of things by noting that a container is
+     either dynamic or not, as a whole, and a given iterator can only iterate
+     over one kind of thing at once: so we can overlap the DTD and non-DTD
+     members, and the structure, variable and enum members, etc.  */
+  union
+  {
+    const ctf_member_t *ctn_mp;
+    const ctf_lmember_t *ctn_lmp;
+    const ctf_dmdef_t *ctn_dmd;
+    const ctf_enum_t *ctn_en;
+    const ctf_dvdef_t *ctn_dvd;
+  } u;
+  /* This union is of various sorts of container we can iterate over:
+     currently dictionaries and archives.  */
+  union
+  {
+    const ctf_file_t *ctn_fp;
+    const ctf_archive_t *ctn_arc;
+  } cu;
+};
+
 /* Return x rounded up to an alignment boundary.
    eg, P2ROUNDUP(0x1234, 0x100) == 0x1300 (0x13*align)
    eg, P2ROUNDUP(0x5600, 0x100) == 0x5600 (0x56*align)  */
@@ -362,6 +392,8 @@  extern ctf_id_t ctf_lookup_by_rawname (ctf_file_t *, int, const char *);
 extern ctf_id_t ctf_lookup_by_rawhash (ctf_file_t *, ctf_names_t *, const char *);
 extern void ctf_set_ctl_hashes (ctf_file_t *);
 
+extern ctf_file_t *ctf_get_dict (ctf_file_t *fp, ctf_id_t type);
+
 typedef unsigned int (*ctf_hash_fun) (const void *ptr);
 extern unsigned int ctf_hash_integer (const void *ptr);
 extern unsigned int ctf_hash_string (const void *ptr);
diff --git a/libctf/ctf-lookup.c b/libctf/ctf-lookup.c
index d95e53706c1..8daab632dca 100644
--- a/libctf/ctf-lookup.c
+++ b/libctf/ctf-lookup.c
@@ -305,6 +305,18 @@  ctf_lookup_by_symbol (ctf_file_t *fp, unsigned long symidx)
   return type;
 }
 
+/* Return the native dict of a given type: if called on a child and the
+   type is in the parent, return the parent.  Needed if you plan to access
+   the type directly, without using the API.  */
+ctf_file_t *
+ctf_get_dict (ctf_file_t *fp, ctf_id_t type)
+{
+    if ((fp->ctf_flags & LCTF_CHILD) && LCTF_TYPE_ISPARENT (fp, type))
+      return fp->ctf_parent;
+
+    return fp;
+}
+
 /* Return the pointer to the internal CTF type data corresponding to the
    given type ID.  If the ID is invalid, the function returns NULL.
    This function is not exported outside of the library.  */
@@ -312,17 +324,16 @@  ctf_lookup_by_symbol (ctf_file_t *fp, unsigned long symidx)
 const ctf_type_t *
 ctf_lookup_by_id (ctf_file_t **fpp, ctf_id_t type)
 {
-  ctf_file_t *fp = *fpp;	/* Caller passes in starting CTF container.  */
+  ctf_file_t *fp = *fpp;	/* Caller passes in starting CTF dict.  */
   ctf_id_t idx;
 
-  if ((fp->ctf_flags & LCTF_CHILD) && LCTF_TYPE_ISPARENT (fp, type)
-      && (fp = fp->ctf_parent) == NULL)
+  if ((fp = ctf_get_dict (fp, type)) == NULL)
     {
       (void) ctf_set_errno (*fpp, ECTF_NOPARENT);
       return NULL;
     }
 
-  /* If this container is writable, check for a dynamic type.  */
+  /* If this dict is writable, check for a dynamic type.  */
 
   if (fp->ctf_flags & LCTF_RDWR)
     {
diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index 93967c6aa8d..d7ed0e31dca 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -18,6 +18,7 @@ 
    <http://www.gnu.org/licenses/>.  */
 
 #include <ctf-impl.h>
+#include <assert.h>
 #include <string.h>
 
 /* Determine whether a type is a parent or a child.  */
@@ -103,6 +104,125 @@  ctf_member_iter (ctf_file_t *fp, ctf_id_t type, ctf_member_f *func, void *arg)
   return 0;
 }
 
+/* Iterate over the members of a STRUCT or UNION, returning each member's
+   offset and optionally name and member type in turn.  On end-of-iteration,
+   returns -1.  */
+
+ssize_t
+ctf_member_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
+		 const char **name, ctf_id_t *membtype)
+{
+  ctf_file_t *ofp = fp;
+  uint32_t kind;
+  ssize_t offset;
+  ctf_next_t *i = *it;
+
+  if (!i)
+    {
+      const ctf_type_t *tp;
+      ctf_dtdef_t *dtd;
+
+      if ((type = ctf_type_resolve (fp, type)) == CTF_ERR)
+	return -1;			/* errno is set for us.  */
+
+      if ((tp = ctf_lookup_by_id (&fp, type)) == NULL)
+	return -1;			/* errno is set for us.  */
+
+      if ((i = ctf_next_create ()) == NULL)
+	return ctf_set_errno (ofp, ENOMEM);
+      i->cu.ctn_fp = ofp;
+
+      (void) ctf_get_ctt_size (fp, tp, &i->ctn_size,
+			       &i->ctn_increment);
+      kind = LCTF_INFO_KIND (fp, tp->ctt_info);
+
+      if (kind != CTF_K_STRUCT && kind != CTF_K_UNION)
+	{
+	  ctf_next_destroy (i);
+	  return (ctf_set_errno (ofp, ECTF_NOTSOU));
+	}
+
+      dtd = ctf_dynamic_type (fp, type);
+      i->ctn_iter_fun = (void (*) (void)) ctf_member_next;
+
+      /* We depend below on the RDWR state indicating whether the DTD-related
+	 fields or the DMD-related fields have been initialized.  */
+
+      assert ((dtd && (fp->ctf_flags & LCTF_RDWR))
+	      || (!dtd && (!(fp->ctf_flags & LCTF_RDWR))));
+
+      if (dtd == NULL)
+	{
+	  i->ctn_n = LCTF_INFO_VLEN (fp, tp->ctt_info);
+
+	  if (i->ctn_size < CTF_LSTRUCT_THRESH)
+	    i->u.ctn_mp = (const ctf_member_t *) ((uintptr_t) tp +
+						  i->ctn_increment);
+	  else
+	    i->u.ctn_lmp = (const ctf_lmember_t *) ((uintptr_t) tp +
+						    i->ctn_increment);
+	}
+      else
+	i->u.ctn_dmd = ctf_list_next (&dtd->dtd_u.dtu_members);
+
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_member_next != i->ctn_iter_fun)
+    return (ctf_set_errno (ofp, ECTF_NEXT_WRONGFUN));
+
+  if (ofp != i->cu.ctn_fp)
+    return (ctf_set_errno (ofp, ECTF_NEXT_WRONGFP));
+
+  /* Resolve to the native dict of this type.  */
+  if ((fp = ctf_get_dict (ofp, type)) == NULL)
+    return (ctf_set_errno (ofp, ECTF_NOPARENT));
+
+  if (!(fp->ctf_flags & LCTF_RDWR))
+    {
+      if (i->ctn_n == 0)
+	goto end_iter;
+
+      if (i->ctn_size < CTF_LSTRUCT_THRESH)
+	{
+	  if (name)
+	    *name = ctf_strptr (fp, i->u.ctn_mp->ctm_name);
+	  if (membtype)
+	    *membtype = i->u.ctn_mp->ctm_type;
+	  offset = i->u.ctn_mp->ctm_offset;
+	  i->u.ctn_mp++;
+	}
+      else
+	{
+	  if (name)
+	    *name = ctf_strptr (fp, i->u.ctn_lmp->ctlm_name);
+	  if (membtype)
+	    *membtype = i->u.ctn_lmp->ctlm_type;
+	  offset = (unsigned long) CTF_LMEM_OFFSET (i->u.ctn_lmp);
+	  i->u.ctn_lmp++;
+	}
+      i->ctn_n--;
+    }
+  else
+    {
+      if (i->u.ctn_dmd == NULL)
+	goto end_iter;
+      if (name)
+	*name = i->u.ctn_dmd->dmd_name;
+      if (membtype)
+	*membtype = i->u.ctn_dmd->dmd_type;
+      offset = i->u.ctn_dmd->dmd_offset;
+      i->u.ctn_dmd = ctf_list_next (i->u.ctn_dmd);
+    }
+
+  return offset;
+
+ end_iter:
+  ctf_next_destroy (i);
+  *it = NULL;
+  return ctf_set_errno (ofp, ECTF_NEXT_END);
+}
+
 /* Iterate over the members of an ENUM.  We pass the string name and associated
    integer value of each enum element to the specified callback function.  */
 
@@ -154,8 +274,126 @@  ctf_enum_iter (ctf_file_t *fp, ctf_id_t type, ctf_enum_f *func, void *arg)
   return 0;
 }
 
+/* Iterate over the members of an enum TYPE, returning each enumerand's NAME or
+   NULL at end of iteration or error, and optionally passing back the
+   enumerand's integer VALue.  */
+
+const char *
+ctf_enum_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
+	       int *val)
+{
+  ctf_file_t *ofp = fp;
+  uint32_t kind;
+  const char *name;
+  ctf_next_t *i = *it;
+
+  if (!i)
+    {
+      const ctf_type_t *tp;
+      ctf_dtdef_t *dtd;
+
+      if ((type = ctf_type_resolve_unsliced (fp, type)) == CTF_ERR)
+	return NULL;			/* errno is set for us.  */
+
+      if ((tp = ctf_lookup_by_id (&fp, type)) == NULL)
+	return NULL;			/* errno is set for us.  */
+
+      if ((i = ctf_next_create ()) == NULL)
+	{
+	  ctf_set_errno (ofp, ENOMEM);
+	  return NULL;
+	}
+      i->cu.ctn_fp = ofp;
+
+      (void) ctf_get_ctt_size (fp, tp, NULL,
+			       &i->ctn_increment);
+      kind = LCTF_INFO_KIND (fp, tp->ctt_info);
+
+      if (kind != CTF_K_ENUM)
+	{
+	  ctf_next_destroy (i);
+	  ctf_set_errno (ofp, ECTF_NOTENUM);
+	  return NULL;
+	}
+
+      dtd = ctf_dynamic_type (fp, type);
+      i->ctn_iter_fun = (void (*) (void)) ctf_enum_next;
+
+      /* We depend below on the RDWR state indicating whether the DTD-related
+	 fields or the DMD-related fields have been initialized.  */
+
+      assert ((dtd && (fp->ctf_flags & LCTF_RDWR))
+	      || (!dtd && (!(fp->ctf_flags & LCTF_RDWR))));
+
+      if (dtd == NULL)
+	{
+	  i->ctn_n = LCTF_INFO_VLEN (fp, tp->ctt_info);
+
+	  i->u.ctn_en = (const ctf_enum_t *) ((uintptr_t) tp +
+					      i->ctn_increment);
+	}
+      else
+	i->u.ctn_dmd = ctf_list_next (&dtd->dtd_u.dtu_members);
+
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_enum_next != i->ctn_iter_fun)
+    {
+      ctf_set_errno (ofp, ECTF_NEXT_WRONGFUN);
+      return NULL;
+    }
+
+  if (ofp != i->cu.ctn_fp)
+    {
+      ctf_set_errno (ofp, ECTF_NEXT_WRONGFP);
+      return NULL;
+    }
+
+  /* Resolve to the native dict of this type.  */
+  if ((fp = ctf_get_dict (ofp, type)) == NULL)
+    {
+      ctf_set_errno (ofp, ECTF_NOPARENT);
+      return NULL;
+    }
+
+  if (!(fp->ctf_flags & LCTF_RDWR))
+    {
+      if (i->ctn_n == 0)
+	goto end_iter;
+
+      name = ctf_strptr (fp, i->u.ctn_en->cte_name);
+      if (val)
+	*val = i->u.ctn_en->cte_value;
+      i->u.ctn_en++;
+      i->ctn_n--;
+    }
+  else
+    {
+      if (i->u.ctn_dmd == NULL)
+	goto end_iter;
+
+      name = i->u.ctn_dmd->dmd_name;
+      if (val)
+	*val = i->u.ctn_dmd->dmd_value;
+      i->u.ctn_dmd = ctf_list_next (i->u.ctn_dmd);
+    }
+
+  return name;
+
+ end_iter:
+  ctf_next_destroy (i);
+  *it = NULL;
+  ctf_set_errno (ofp, ECTF_NEXT_END);
+  return NULL;
+}
+
 /* Iterate over every root (user-visible) type in the given CTF container.
-   We pass the type ID of each type to the specified callback function.  */
+   We pass the type ID of each type to the specified callback function.
+
+   Does not traverse parent types: you have to do that explicitly.  This is by
+   design, to avoid traversing them more than once if traversing many children
+   of a single parent.  */
 
 int
 ctf_type_iter (ctf_file_t *fp, ctf_type_f *func, void *arg)
@@ -175,7 +413,11 @@  ctf_type_iter (ctf_file_t *fp, ctf_type_f *func, void *arg)
 }
 
 /* Iterate over every type in the given CTF container, user-visible or not.
-   We pass the type ID of each type to the specified callback function.  */
+   We pass the type ID of each type to the specified callback function.
+
+   Does not traverse parent types: you have to do that explicitly.  This is by
+   design, to avoid traversing them more than once if traversing many children
+   of a single parent.  */
 
 int
 ctf_type_iter_all (ctf_file_t *fp, ctf_type_all_f *func, void *arg)
@@ -195,6 +437,55 @@  ctf_type_iter_all (ctf_file_t *fp, ctf_type_all_f *func, void *arg)
   return 0;
 }
 
+/* Iterate over every type in the given CTF container, optionally including
+   non-user-visible types, returning each type ID and hidden flag in turn.
+   Returns CTF_ERR on end of iteration or error.
+
+   Does not traverse parent types: you have to do that explicitly.  This is by
+   design, to avoid traversing them more than once if traversing many children
+   of a single parent.  */
+
+ctf_id_t
+ctf_type_next (ctf_file_t *fp, ctf_next_t **it, int *flag, int want_hidden)
+{
+  ctf_next_t *i = *it;
+
+  if (!i)
+    {
+      if ((i = ctf_next_create ()) == NULL)
+	return ctf_set_errno (fp, ENOMEM);
+
+      i->cu.ctn_fp = fp;
+      i->ctn_type = 1;
+      i->ctn_iter_fun = (void (*) (void)) ctf_type_next;
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_type_next != i->ctn_iter_fun)
+    return (ctf_set_errno (fp, ECTF_NEXT_WRONGFUN));
+
+  if (fp != i->cu.ctn_fp)
+    return (ctf_set_errno (fp, ECTF_NEXT_WRONGFP));
+
+  while (i->ctn_type <= fp->ctf_typemax)
+    {
+      const ctf_type_t *tp = LCTF_INDEX_TO_TYPEPTR (fp, i->ctn_type);
+
+      if ((!want_hidden) && (!LCTF_INFO_ISROOT (fp, tp->ctt_info)))
+	{
+	  i->ctn_type++;
+	  continue;
+	}
+
+      if (flag)
+	*flag = LCTF_INFO_ISROOT (fp, tp->ctt_info);
+      return LCTF_INDEX_TO_TYPE (fp, i->ctn_type++, fp->ctf_flags & LCTF_CHILD);
+    }
+  ctf_next_destroy (i);
+  *it = NULL;
+  return ctf_set_errno (fp, ECTF_NEXT_END);
+}
+
 /* Iterate over every variable in the given CTF container, in arbitrary order.
    We pass the name of each variable to the specified callback function.  */
 
@@ -229,6 +520,63 @@  ctf_variable_iter (ctf_file_t *fp, ctf_variable_f *func, void *arg)
   return 0;
 }
 
+/* Iterate over every variable in the given CTF container, in arbitrary order,
+   returning the name and type of each variable in turn.  The name argument is
+   not optional.  Returns CTF_ERR on end of iteration or error.  */
+
+ctf_id_t
+ctf_variable_next (ctf_file_t *fp, ctf_next_t **it, const char **name)
+{
+  ctf_next_t *i = *it;
+
+  if ((fp->ctf_flags & LCTF_CHILD) && (fp->ctf_parent == NULL))
+    return (ctf_set_errno (fp, ECTF_NOPARENT));
+
+  if (!i)
+    {
+      if ((i = ctf_next_create ()) == NULL)
+	return ctf_set_errno (fp, ENOMEM);
+
+      i->cu.ctn_fp = fp;
+      i->ctn_iter_fun = (void (*) (void)) ctf_variable_next;
+      if (fp->ctf_flags & LCTF_RDWR)
+	i->u.ctn_dvd = ctf_list_next (&fp->ctf_dvdefs);
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_variable_next != i->ctn_iter_fun)
+    return (ctf_set_errno (fp, ECTF_NEXT_WRONGFUN));
+
+  if (fp != i->cu.ctn_fp)
+    return (ctf_set_errno (fp, ECTF_NEXT_WRONGFP));
+
+  if (!(fp->ctf_flags & LCTF_RDWR))
+    {
+      if (i->ctn_n >= fp->ctf_nvars)
+	goto end_iter;
+
+      *name = ctf_strptr (fp, fp->ctf_vars[i->ctn_n].ctv_name);
+      return fp->ctf_vars[i->ctn_n++].ctv_type;
+    }
+  else
+    {
+      ctf_id_t id;
+
+      if (i->u.ctn_dvd == NULL)
+	goto end_iter;
+
+      *name = i->u.ctn_dvd->dvd_name;
+      id = i->u.ctn_dvd->dvd_type;
+      i->u.ctn_dvd = ctf_list_next (i->u.ctn_dvd);
+      return id;
+    }
+
+ end_iter:
+  ctf_next_destroy (i);
+  *it = NULL;
+  return ctf_set_errno (fp, ECTF_NEXT_END);
+}
+
 /* Follow a given type through the graph for TYPEDEF, VOLATILE, CONST, and
    RESTRICT nodes until we reach a "base" type node.  This is useful when
    we want to follow a type ID to a node that has members or a size.  To guard
diff --git a/libctf/ctf-util.c b/libctf/ctf-util.c
index 1f745f533b8..5abea6a70bb 100644
--- a/libctf/ctf-util.c
+++ b/libctf/ctf-util.c
@@ -173,3 +173,32 @@  ctf_set_errno (ctf_file_t * fp, int err)
   fp->ctf_errno = err;
   return CTF_ERR;
 }
+
+/* Create a ctf_next_t.  */
+
+ctf_next_t *
+ctf_next_create (void)
+{
+  return calloc (1, sizeof (struct ctf_next));
+}
+
+/* Destroy a ctf_next_t, for early exit from iterators.  */
+
+void
+ctf_next_destroy (ctf_next_t *i)
+{
+  free (i);
+}
+
+/* Copy a ctf_next_t.  */
+
+ctf_next_t *
+ctf_next_copy (ctf_next_t *i)
+{
+  ctf_next_t *i2;
+
+  if ((i2 = ctf_next_create()) == NULL)
+    return NULL;
+  memcpy (i2, i, sizeof (struct ctf_next));
+  return i2;
+}
diff --git a/libctf/libctf.ver b/libctf/libctf.ver
index 93616d83691..e99f890e0e1 100644
--- a/libctf/libctf.ver
+++ b/libctf/libctf.ver
@@ -73,6 +73,7 @@  LIBCTF_1.0 {
 	ctf_type_compat;
 
 	ctf_member_info;
+	ctf_member_next;
 	ctf_array_info;
 	ctf_member_count;
 
@@ -87,10 +88,17 @@  LIBCTF_1.0 {
 
 	ctf_member_iter;
 	ctf_enum_iter;
+	ctf_enum_next;
 	ctf_type_iter;
+	ctf_type_next;
 	ctf_type_iter_all;
 	ctf_label_iter;
 	ctf_variable_iter;
+	ctf_variable_next;
+
+	ctf_next_create;
+	ctf_next_destroy;
+	ctf_next_copy;
 
 	ctf_add_array;
 	ctf_add_const;
@@ -138,6 +146,7 @@  LIBCTF_1.0 {
 	ctf_arc_open_by_name_sections;
 	ctf_archive_count;
 	ctf_archive_iter;
+	ctf_archive_next;
 	ctf_archive_raw_iter;
 	ctf_get_arc;