PAR scope and direction

Discuss the generic proposals for SJTAG
User avatar
Bradford Van Treuren
SJTAG Chair Emeritus
Posts: 103
Joined: Fri Nov 16, 2007 2:06 pm
Location: NOKIA / USA

Re: PAR scope and direction

Post by Bradford Van Treuren » Wed Feb 28, 2018 10:58 pm

I have tried to point out the distinction between "test coverage" and "test access" on a number of occasions. "Test coverage" has to do with a particular test or set of tests applied to a target UUT. Whereas, "test access" has to do with the interface to the unit being tested. Now there seems to be a real question on what is defined as the "interface" of the UUT. I hear Terry talking about interface being a set of API calls defining the test capabilities a given UUT advertises to be able to perform diagnostics on that UUT. That is certainly one form of "interface." I hear Louis talking about the need for defining what a UUT must provide as this API to be compliant to a standard. I find the latter much more difficult based on my first point. That set of API methods is heavily dependent on what tests and applications are being applied to the UUT through that API. Thus, the coverage of the UUT is dependent on what the test is targeting. That is very specific to what that test is designed for. A memory test is going to have a very different coverage profile from an interconnect test or a power monitor probe. You can't cookie cut "test coverage" into a generic rubber stamp; that is highly dependent on the application being performed to the UUT.

I understand what Terry is getting at for a System to Sub-system interface. However, like Ian described, those interfaces are well defined in my systems because I have more control of the system architecture. I get no real value add from such a discussion for my needs. It is not so say these points are not needed for some sectors of industry, but they are certainly not for all. In Mobility industry, there are consortiums of vendors and customers who define such interfaces, in great detail, between architectural blocks in the systems used by the Telecommunications Industry providers. The most well known is the consortium of 3GPP. Standard interfaces, such as CPRI, define control, data, and maintenance operations to be handled between the various modules. This allows vendor A's BBU to plug-n-play with vendor B's Radio Antenna, for example. Similarly, the Automotive Industry, has consortiums defining standardized interfaces between their architectural blocks. ODB-II being one such interface.

Last week, I asked how the topic of "test coverage" or even defining a System to Sub-system API was going to solve my need for coordinating and accessing these various standard HW interfaces (1687, 1149.1-2013, I2C, SPI, etc.) within a board design? There was no response. I explained how the current standards end at the edge of the devices. There is a gap in support for leveraging such tooling/instrumentation for board and system level testing. When I did my first embedded boundary-scan application in 1990, people thought I was crazy because it was a manufacturing test technology. I had to convince people that those same tests used in the factory could in-fact be leveraged for in-system testing with the right hardware and software in place. The rest is history as it is now very common place in advanced systems. SJTAG has the same potential for growth and ROI.

The existing standards will not be used outside of chip testing if there is no way to coordinate and manage these "interfaces" to these standards. The term "interface" here is multi-fold. There is a diverse set of HW interfaces mating to each of these standards. There is also a diverse set of protocols used to communicate with these standards. How does one communicate with each of these standards? Boundary-Scan Tool vendors have provided separate applications in the past for different "interface/application" combinations being used from a boundary-scan bit bang emulation bridge. For example, a SPI interface application is different depending on what is connected to it. A power manager has certain operations that have to be performed that are different from a SPI FLASH. Even worse, vendor A power manager has a totally different dictionary of commands and possibly a different protocol from Vendor B. Generic SPI write and read operations are typically provided as primitives, but applications define the protocol as to how that SPI bus is used and the order of the data being sent and read. Where does "test coverage" come into play on this? It really doesn't. One could argue the application of the data is performing some form of coverage of the circuit. However, keep in mind a power manager application will not program a SPI FLASH. There has to be some way of describing the algorithms needed to perform the target operations (the protocol) as well as defining what kind of bus is being used. Currently, each vendor provides their own model of the protocol as a proprietary description of the HW and statements to be applied with assumptions to the data format being applied. It is purely ad hoc and not interoperable between different busses performing the same instrument function. In most cases, the model describing one application on a bus will be different for a second application over the same bus. This is quite costly for tool vendors to support as they need to reinvent a new ad hoc model and code for each new interface that does not match the current assumptions exactly. With the vendor tools I have used, I have seen them evolve over time as technology changes and busses getting more complex (e.g., SPI, DSPI, QSPI). Worse yet, I have to wait for that tooling technology to be developed to even be able to use it for my product. With 1687 and 1149.1-2013, these standards provide the HW description (ICL/BSDL) and protocol algorithms (PDL), but scoped to the single instrument being used. There is nothing defined for coordinating them at the board level and managing concurrency of operation. Retargeters are available for retargeting the data to the device edge, but not beyond that.

What does this have to do with System Test Access Management (STAM) as a standard? Everything! If I can't get access to registers inside devices, I can't produce product. As designs get more complex and systems become SoCs, the HW interface pins are going to be a premium as there will be more IO and power pins then control pins allowed. This evolution is why standards like 1581, 1149.1-2013, and 1687 were developed in the first place. If these standards are going to be used for functional registers (registers used for normal operation of the device), there has to be some way to automate the access to these registers from the embedded software (a smart driver/scheduler). The same mechanism is useful for external testing or control/monitoring from a System Interface connection. What makes this part difficult is the necessity for retargeting of data being applied to different hierarchical levels of a particular device. This is really no different than dealing with retargeting of data at the board and system levels. It is just a hierarchy. Where it gets tricky is where the data spans between one bus domain to another (a bridge). That could be 1149.1 to I2C via an I2C master instrument controlled with 1149.1 or between a System and a Sub-system. Transformation of the data and protocol must be accomplished for these bridges. Terry stated a Sub-system needs to advertise what features/methods are available for use by the hosting System. I would argue that STAM is one of many such methods available for System Access. IPMI tests are others for ATCA environments. STAM has to deal with the interoperability between different standards on a board and how that ripples up to the System/Sub-system interface as just another standard bridge definition.

The Future:
Where is all this going? I see STAM, as I described, being an important enabling technology for future systems that are going to be based on Machine Learning algorithms that have to dynamically read sensors and manipulate controls in an automated fashion. Without a standardized way to coordinate access to sensors and controls, these kinds of systems will be difficult to create. Most likely industry will bypass the standards bodies and implement their own solutions through their own consortiums like they have done in the past.
Bradford Van Treuren
Distinguished Member of Technical Staff
NOKIA MN

User avatar
Bradford Van Treuren
SJTAG Chair Emeritus
Posts: 103
Joined: Fri Nov 16, 2007 2:06 pm
Location: NOKIA / USA

Re: PAR scope and direction

Post by Bradford Van Treuren » Thu Mar 01, 2018 6:35 pm

One hobby I have is studying Domain Specific Languages (DSL) and how people design these languages for particular use cases. I stumbled across one that is open source targeting at first Networking Domains, but claims it is adaptable to any configurable processing stream. It shows a good example of a "top down" processing of data passing from point A to point B through a Network. For STAM, think of point A being the application at the top and many point Bs being the target registers inside devices with some sort of routing that takes place in between.

Top Down Processing Example
The DSL I was looking at is called P4 and is being defined by a consortium of vendors, customers, silicon vendors, and academia. The information on P4 can be found at: https://github.com/p4lang/tutorials.

The basic premise is data comes in from the left and is parsed to determine what type of data it is. The data has key fields that can be used to identify what the routing needs to be based on the type of data being processed. The data stream parses the input meta-data and passes it on to the appropriate agent who has a Match Table and associated Action Macros for each match to process/transform the meta-data and route it to the next stage or appropriate queue.
Image
To put this into real world terms, the parser has a defined set of protocols it understands and is able to parse the input data based on those set protocols. If none match, it passes it to a default queue stream to be processed by someone else. If a protocol is detected, the data is handed off to the appropriate agent pipeline.
Image
Image
Example P4 descriptions that get compiled into the end product's native code are shown here:
Image

Bottom Up Processing Example
In contrast to that, IEEE P1687.1 is looking at a bottom up approach where callbacks are defined to request transformation of the data required by a lower level into directives able to be sent to the next higher level in order to produce such a data stimulus to the lower level. The purpose of this bottom up concept is because the top level application is ultimately wanting to change the state of some register in the system. There exists a description of that register in the device and code describing how to access a particular behavior for an instrument that uses such a register. So only the bottom level knows what has to be sent based on the only description available in order to provide such a behavior. That then has to ripple up the hierarchy to the top level where the particular stimulus can be applied to drive the requested behavior change at the lower level. Below is a diagram currently used by the P1687.1 working group to describe this process.
Image

So here are two different approaches with different Domain Specific Languages trying to solve the problem of sending data from point A to point B through interfaces that have to transform the structure of the message as it passes through each node in the hierarchy. This is a good example of what I have been trying to describe as a huge gap in the System Test Access Management domain within a board and between systems and sub-systems. It is not a trivial problem and needs to be open enough to allow any protocol to be used for transmission of that data. There is an access component to each of these mapping a pattern to a destination as well. In the first, the meta-data must contain addressing information for the current protocol being processed. In the second, the callback is assembling the appropriate actions required to be applied to the left side that will ensure the correct addressing of the right hand side element as part of a particular atomic transformation. The advantage of the second approach is it does not require a packetized protocol to be used. Thus, only the transmission of data is required based on directives being applied in the right order. This also simplifies the information needing to be stored to map the leaf bits from the instruments to top level bits in order to provide a means for diagnosis. This second approach seems to be a better fit for what STAM needs to define at the board and system levels mainly because it does not require a packet of data with a set format to be processed. This leaves it open for any protocol to be applied: packetized or not.
Bradford Van Treuren
Distinguished Member of Technical Staff
NOKIA MN

User avatar
Ian McIntosh
SJTAG Chair
Posts: 415
Joined: Mon Nov 05, 2007 11:49 pm
Location: Leonardo, UK
Contact:

Re: PAR scope and direction

Post by Ian McIntosh » Sun Mar 04, 2018 3:34 pm

I'm only just getting around to looking at Brad's last couple of posts - I seem to have spent the last four days shovelling snow and haven't made it in to work since Wednesday.

On first reading, I'm not sure if Brad is arguing for the P4 top-down approach or the P1687.1 bottom-up approach or just contrasting the two. But I feel it is probably a more complex scenario: the P1687.1 description acknowledges that the application (at the top level) wants to do something - it has an intent - so that somehow has to be propagated as a chain of lower level actions. Without thinking about it too hard, there must be knowledge at the top level of what the target leafs at the bottom can do, and that has to be rippled up the way (there's a bit of an assumption that you're trying to automate some of the process here - if you're hand-carving a test you kind of assemble all the information without really thinking about any kind of information "flow"). I think you maybe need/want to also assume a lack of sentience as you pass through levels, i.e. the application level needs to know what transactions are required all the way down the stack. So you can consider that the "command flow" at least is top-down. It may be that once you've set up the pathway for given operation you may be able to consider the movement of the actual data as almost peer-to-peer. :?
Ian McIntosh
Testability Lead
Leonardo MW Ltd.

User avatar
Ian McIntosh
SJTAG Chair
Posts: 415
Joined: Mon Nov 05, 2007 11:49 pm
Location: Leonardo, UK
Contact:

Re: PAR scope and direction

Post by Ian McIntosh » Sun Mar 04, 2018 6:34 pm

In retrospect, perhaps this top-down vs bottom-up discussion is really implementational and not really giving us direction on completing a PAR.

We seem to keep going round in circles on Need and I'm now feeling that the reason (or a reason) for that is that we don't all agree on the fundamental scope of the project, so I've adjusted the agenda for the meeting on March 5th to start by looking at that.
Ian McIntosh
Testability Lead
Leonardo MW Ltd.

User avatar
Ian McIntosh
SJTAG Chair
Posts: 415
Joined: Mon Nov 05, 2007 11:49 pm
Location: Leonardo, UK
Contact:

Re: PAR scope and direction

Post by Ian McIntosh » Mon Mar 12, 2018 6:57 pm

The draft Scope for STAM, as it stands is:
SCOPE:
This standard defines methods to allow, in conjunction with existing methods, for the coordination and control of device, board, and sub-system test interfaces to extend access to the system level. The standard does not replace or provide an alternative to existing test interface standards, but aims instead to leverage them by defining a description to better manage how they are used in the system.
Can we refine this? Is the language "accessible"? Is there anything missing? Anything misleading?
Ian McIntosh
Testability Lead
Leonardo MW Ltd.

User avatar
Bradford Van Treuren
SJTAG Chair Emeritus
Posts: 103
Joined: Fri Nov 16, 2007 2:06 pm
Location: NOKIA / USA

Re: PAR scope and direction

Post by Bradford Van Treuren » Mon Mar 19, 2018 4:08 pm

Here are the edits to the Scope section as described today:

This standard describes defines methods to allow, in conjunction with existing methods, for the coordination and control of a variety of interfaces to devices, boards, and sub-systems test interfaces to extend test access to the system level. The standard does not replace or provide an alternative to existing test interface standards, but aims instead to enable their usage throughout the hierarchy of the system. leverage them by defining a description to better manage how they are used in the system.


A clean (not marked up) copy of the proposed standard is listed below:

This standard describes methods to allow, in conjunction with existing methods, for the coordination and control of a variety of interfaces to devices, boards, and sub-systems to extend test access to the system level. The standard does not replace or provide an alternative to existing test interface standards, but aims instead to enable their usage throughout the hierarchy of the system.
Bradford Van Treuren
Distinguished Member of Technical Staff
NOKIA MN

TerryD
SJTAG Member
Posts: 3
Joined: Mon Sep 18, 2017 4:23 pm

Re: PAR scope and direction

Post by TerryD » Mon Mar 19, 2018 8:40 pm

Print it!

User avatar
Joel Irby
SJTAG Member
Posts: 3
Joined: Mon Sep 18, 2017 4:12 pm

Re: PAR scope and direction

Post by Joel Irby » Mon Mar 19, 2018 9:51 pm

I agree, this is a good statement of the scope of STAM.

User avatar
Heiko Ehrenberg
SJTAG Vice Chair
Posts: 45
Joined: Wed Nov 21, 2007 3:15 pm
Location: GOEPEL Electronics - Austin, TX
Contact:

Re: PAR scope and direction

Post by Heiko Ehrenberg » Tue Mar 20, 2018 2:13 pm

Looks good to me.
- Heiko

User avatar
Ian McIntosh
SJTAG Chair
Posts: 415
Joined: Mon Nov 05, 2007 11:49 pm
Location: Leonardo, UK
Contact:

Re: PAR scope and direction

Post by Ian McIntosh » Tue Mar 20, 2018 7:46 pm

It seems fine although I do have a slight reservation...
... coordination and control of a variety of interfaces to devices, ...
The underlined words "feel" a little weak to me. I think it's certainly true that we're looking at the existence of different interface types - if the system only used one interface type, e.g. 1149.1, then you could probably get by without STAM. So the reader ought to be able to infer that we mean schemes that involve more than one interface type, and I'm just not sure that "a variety of" really does that - you could read it as "one of many" rather than "more than one". On the other hand, I really don't like the option of "a plurality of" that seems to be preferred in writing patents.

Just throwing in "multiple" doesn't work either as it might imply that each device has multiple interfaces (they may, but they need not).

I'd like a better option here, but I'll accept what we have if we can't find one.
Ian McIntosh
Testability Lead
Leonardo MW Ltd.

User avatar
Bradford Van Treuren
SJTAG Chair Emeritus
Posts: 103
Joined: Fri Nov 16, 2007 2:06 pm
Location: NOKIA / USA

Re: PAR scope and direction

Post by Bradford Van Treuren » Tue Mar 20, 2018 11:45 pm

... coordination and control of a variety of interfaces to devices, ...
The other legalese term used often is "a multiplicity of." That has the same problem as "multiple." Other terms that might work are:
  • an assortment of
  • a diversity of
  • a conglomeration of
  • a combination of
The reality is I find most often it is a mishmash of interfaces I have to deal with.
Bradford Van Treuren
Distinguished Member of Technical Staff
NOKIA MN

User avatar
Adam W Ley
SJTAG Established Member
Posts: 33
Joined: Mon Nov 12, 2007 1:35 pm
Location: ASSET InterTech, Inc./ USA
Contact:

Re: PAR scope and direction

Post by Adam W Ley » Wed Mar 21, 2018 1:24 am

"mishmash" says it well to me, too, though maybe its not universally translatable.

While I can see Ian's point, "variety" may be the best since it invokes the idea of multiplicity of type in addition to multiplicity of number ...

User avatar
Adam W Ley
SJTAG Established Member
Posts: 33
Joined: Mon Nov 12, 2007 1:35 pm
Location: ASSET InterTech, Inc./ USA
Contact:

Re: PAR scope and direction

Post by Adam W Ley » Wed Mar 21, 2018 1:38 am

To the whole, I think I'm aligned, also, though I do wonder about the use of articles in proximity to "system".

Does the following read correctly? Better?
This standard describes methods to allow, in conjunction with existing methods, for the coordination and control of a variety of interfaces to devices, boards, and sub-systems to extend test access to the system level. The standard does not replace or provide an alternative to existing test interface standards, but aims instead to enable their usage throughout the hierarchy of the systems.
I suppose in this rendering that "system" requires some qualification. While "conforming" would be the usual sort of word here, I don't think that we know yet that a "system" would be a conforming unit. (This, of course, begs the very important question as to what would be a conforming "unit"?)

A factual, though wordy, way to say it might be "systems that adopt such methods"?

User avatar
Ian McIntosh
SJTAG Chair
Posts: 415
Joined: Mon Nov 05, 2007 11:49 pm
Location: Leonardo, UK
Contact:

Re: PAR scope and direction

Post by Ian McIntosh » Wed Mar 21, 2018 7:59 pm

Bradford Van Treuren wrote:
Tue Mar 20, 2018 11:45 pm
The reality is I find most often it is a mishmash of interfaces I have to deal with.
"Dog's Breakfast" is the term that comes to mind.

Seriously though, I like Adam's proposed amendments, and I can live with "variety of".
Ian McIntosh
Testability Lead
Leonardo MW Ltd.

User avatar
Bradford Van Treuren
SJTAG Chair Emeritus
Posts: 103
Joined: Fri Nov 16, 2007 2:06 pm
Location: NOKIA / USA

Re: PAR scope and direction

Post by Bradford Van Treuren » Wed Mar 21, 2018 9:10 pm

Adam Ley wrote:
To the whole, I think I'm aligned, also, though I do wonder about the use of articles in proximity to "system".
I am thinking the first article is referring to level and not system. To me removing the first the in Adam's proposal reads a bit awkward. Without the being present, IMHO, it does force the subject of the sentence to system and not level.

Are we not talking about interfaces, which exist at different levels of the hierarchy defining mating points (from a control and management perspective) between devices, boards, and systems? These interfaces are themselves standardized adapters providing a means of capturing or altering the state of the entity hosting the interface - be it a device, board or system.

So I am more inclined to have the wording be:

This standard describes methods to allow, in conjunction with existing methods, for the coordination and control of a variety of interfaces to devices, boards, and sub-systems to extend test access to the system level. The standard does not replace or provide an alternative to existing test interface standards, but aims instead to enable their usage throughout the hierarchy of systems that adopt such methods.
Bradford Van Treuren
Distinguished Member of Technical Staff
NOKIA MN

Post Reply