The discussion forum for the SJTAG Initiative Group

SJTAG definition of a "system"

Discuss the generic proposals for SJTAG
User avatar
Ian McIntosh
SJTAG Chair
Posts: 354
Joined: Mon Nov 05, 2007 11:49 pm
Location: Leonardo MW Ltd, UK
Contact:

SJTAG definition of a "system"

Postby Ian McIntosh » Sun Oct 08, 2017 6:19 pm

I'm going to attempt to capture the very useful discussion that happened in email in the last week. I'm basing the timeline purely on the order the messages are time-stamped in my email folder, starting with the earliest - that means some responses don't directly follow what they are replying to, but generally I think the thread is followable:
Louis wrote:In pursuit of reworking the legacy SJTAG definition " a system may be described as an organized collection of components or assemblies that are designed to operate together to perform one or more tasks or functions," I would like to comment that some of the functions are "synergistic" (only exist because of the combined contribution of two or more assemblies) but other functions are distinct irrespective of other assemblies in the system. The tests need to take this into account. One Design for Testability goal should be to allow each of the assemblies to be tested in that independent (isolated) manner.

Ian wrote:Indeed Louis, but at the risk of pedantry, the description of the lower level assembly (sub-assembly, board or whatever) would define the intended tasks or functions as only the set that can be provided by that level of assembly. So the definition seems to remain valid in the context of whatever you’re considering to be the testable item.

You’re certainly right that it is often the case that “the whole may be more than the sum of its parts”. That seems to lean towards “functional” test; in my experience, functional tests are generally very good at detecting the presence of a fault but not so good at diagnosing where it lies, unless the tests are very carefully designed and very focussed. And that generally requires that the testability is built-in from the bottom up (in my domain that means starting with the board designs) as trying to ask for more testability once something is built is a tough ask. So yes, the sub-assemblies need to be testable in their own right.

Louis wrote:If we are considering SJTAG (test) interfaces to be based on 1149.1 or 1687 then these mechanisms need to be able to help isolate assemblies from the system as a whole and should be eluded to in our definition of a system. A triangle is not merely a collection of 3 lines, but for testability and diagnosability we should be able to test each line for its properties without regards to the triangle. If all interfaces between the lines (corners and angle of the triangle) have the same testability properties (say, 1149.1 or 1687) then we should be able to accomplish that. I am suggesting that we say so in the definition.

So, actually I am not saying we should alter the properties of the definition that imply that “the whole may be more than the sum of its parts”. What I am trying to say, instead, is that the SJTAG definition should (in addition) encompass our desire to test the assemblies separately.

Terry wrote:It sounds like we are trying to describe a system in a way that guarantees that we will use 1149.1 and 1687 in the solution. My biggest concern is that doing so may limit the ability of whatever standard is eventually defined to be adopted. As we get further up the system hierarchy, the contribution of test programs based on scan chains is significantly reduced. Thus, requiring access be supported up at the highest levels becomes a cost/benefit question where test will often lose out.

In addition, we are trying to bridge a very tools-based testing environment (Chip and ICT board level testing), where pattern based testing is very common, with system level testing where functional testing rules the roost with it’s incumbent interdependency of software and hardware. In addition, we are bridging numerous vendor layers. It seems like if our group’s task is to document the need for a System Level Test Standard, then we must first find out what are the biggest headaches for system test folks. There are some on our committee who work in this field and that is a start. I would also wonder if we could get information from other large system designers to understand what types of problems they are seeing. Or perhaps get a survey of some sort going at conferences such as Autotestcon were there is a large proliferation of system test engineers. Once we have discovered and documented those problems, we can bin them to categories to outline the need. With that information, I think it would be the challenge of the standards committee to see if they can come up with a methodology that would meet those needs. Whether or not existing standards can be leveraged would have a lot to do with what the needs are.

All of this points to making sure that we are not trying to force the solution before we even justify the needs. I believe this means we should keep the definitions more generic and not add too much specific detail into them.

Naveen wrote:I agree. There are repository of past discussion and documents but need some real world problem schematics to look for solution and hence standards. Right now universe looks too big. The need here to write out those problem and looks for solution with standard.

Louis wrote:Great discussion...

Let's not forget that the name of the group is SJTAG, which implies - if it doesn't actually state - that we are trying to apply JTAG to system level test. Unless I am missing something, JTAG is 1149.1 and 1687 (and to lesser degree 1500, 1149.6, 1149.10 and even possibly SPI and I2C). So even if we can avoid these things in the definition of "system" they will pop up in the Scope and/or the Purpose.

I will let our Navy test experts chime in on this, Terry, but in chairing the Testability Panel at AUTOTESTCON for many years, I can tell you that System Level Test for that audience has always centered around being able to diagnose to the responsible replaceable subassembly.

I believe this is no different for other systems. A laptop manufacturer at Dell wants to know which assembly is causing a computer malfunction. When you at National put together an ATE and have a virtual instrument cobbled together from a power supply and a signal generator, you will want to know which one of those two is the culprit if the "system" is not working properly.

Perhaps the definition of the system can bypass the diagnostic issue, but wouldn't it be better to incorporate it there?

Terry wrote:Including the need for a diagnostic capability, yes, certainly that should be included. Trying to force 1149.1 and such all the way up to a system level is that real problem. For example, in my ATE case, I want to know if the power supply or the signal generator is the culprit. If both have provided some level of self-test that I can execute from the system level, that would be sufficient. I would not really care if I triggered this hypothetical self-test through a bscan method or by a special command through PXIe or GPIB. It would serve the purpose of the diagnostic task. That is my point. Let’s identify the behaviors that are needed and not try to define how they are executed.

Ian wrote:"Let's not forget that the name of the group is SJTAG, which implies - if it doesn't actually state - that we are trying to apply JTAG to system level test. Unless I am missing something, JTAG is 1149.1 and 1687 (and to lesser degree 1500, 1149.6, 1149.10 and even possibly SPI and I2C). So even if we can avoid these things in the definition of "system" they will pop up in the Scope and/or the Purpose."
In one word - No!
The title of the group is "System Test Access Management" specifically to get away from that 1149.1 centred mindset (and more pedantically, JTAG, as it was originally conceived, had a much broader consideration than just the serial digital databus that became 1149.1).
"SJTAG" is a convenient short form that ties in the legacy that the current study group grew out of, in much the same way that "JTAG" has stuck as a label.

Ian wrote:Louis, as I now understand your suggestion, you are proposing that a system, for our purposes, should provide means to isolate a sub-assembly from the other parts of the system in order that the sub-assembly might be better tested without interference from other sub-assemblies?
While that's a desirable property, I don't believe it can be a mandatory one. I'd argue that SJTAG could still be applied to a system that does not exhibit such a property, although what it could achieve might be reduced. This broaches on the notion that we could have "SJTAG compliant" and "SJTAG compatible" systems, where a "compatible" system may not have all the features desired for SJTAG but nevertheless supports its use.

Louis wrote:So let me understand:

Per Ian, this group "System Test Access Management" is in and SJTAG is out? So what "test access" mechanism are we considering here?

To Terry: Your approach to do self test of the instrument assemblies is fine, but still we need a way to communicate the results of those tests. 1149.1 or more likely 1687 is designed to be that test access. 1687 had the stated purpose to create hierarchical test information access from ICs to boards and from boards to systems as well as the other direction.

While I am surprised by these revelations of JTAG-less access, it does not change my original point that diagnosability is an important aspect of system level test and should be included in some form in the definition of a system or of a system test.

Ian wrote:I am not saying "JTAG-less", just not "JTAG only". The way we got here is that we recognised that, even if your intention is to perform an 1149.1 type of test, sometimes you need to interact with other interfaces to make that happen. That may be that you need to go outside of the 1149.1 protocols to operate a gateway, or use a parallel IO on some device to enable some part of a circuit or use I2C to make a path selection (I don't recall the details, but Brad had an example of, I think, a Brocade device that used I2C for 1149.1 path selection). If we're trying to organise tests at a level higher than the chip then we need to consider how we manage and coordinate the interfaces involved.
There's a parallel with 1687: I believe 1687 wanted to address more than just an 1149.1 external interface but fell foul of an overly specific statement of scope, hence 1687.1 being created to provide an extension for other serial buses.

Naveen wrote:In my understanding what we are looking for is:

“How to access and manage test (and/or diagnosis) from TOP to BOTTOM given the interface of hierarchical sub system (chip or board or …) which can be utilized (and/or proposed to support the modification required to ease system test and diagnosis)”.

It could be 1149.1 (which is one of interface standard and similar related 1687, 1500 down the level), I2C or SPI etc. The way to access and manage standalone chip is in place with these standard but what constraints and how to communicate and manage at upper system level looks challenge and looking for standard.

Louis wrote:OK - JTAG+.

Regardless of the means, I am convinced that we need the "system test" definition to include the necessity to diagnose unambiguously to a replaceable unit or if necessary, a limited number of replaceable units.

Ian wrote:OK, but a definition for "system test" is different from a definition for "system" - the latter makes no assumption about a use case, which is what the former is.

Ian wrote:I'd agree that is probably a fairly succinct evaluation of what we're aiming to do, Naveen.

Brad wrote:The definition of "system" should not contain the notion of diagnostics. System is a noun. I would agree that "system test" definition may include the notion of diagnostics, but it is not mandatory. SJTAG, as we are discussing it, is "System Test Access Management" as Ian pointed out. The key is "access management" in being able to obtain the information necessary to perform the diagnostics of a system. "System Test" is responsible for sorting through all the resultant data to determine a root cause as the diagnostics. SJTAG is an enabling technology that allows an application performing “system test” to be able to obtain the necessary information to conduct the diagnostic analysis. We are not trying to define a standard for a system tester. We ARE trying to define an enabling technology to allow a system tester to perform its function. So I would claim that SJTAG, based on this team’s title, is diagnostic agnostic in that SJTAG moves data through a system hierarchy, but it does not know for what purpose the data is used for. The data may be a snapshot of an internal state of an ASIC, it may be the result of an MBIST operation, it may be observed values of an interconnect test vector, etc. SJTAG really does not care. It is just responsible to get the data from point A to point B in a standardized, reproducible way. That is, in fact, the mission of 1149.1 and 1687 as well. However, both these standards have a way of preserving intent of the data by associating bit values with particular elements of the physical design (e.g, device pin, instrument register bit) via BSDL and ICL. That preservation is what we need to ensure is propagated back up the hierarchy so that an application at the top knows what part of the data represents the target bits at the leaves. The leveraged standards are responsible for mapping intent to bits at the leaves. SJTAG needs to have a means of preservation of location/position in the resulting data from all the transformations the data passes through in the hierarchy so an application may be able to reconstruct/deconstruct the data in the top level with the data actually at the leaves in order to perform the test/diagnostic functions. BUT this is only for interfaces bridging to these kinds of standards where this data is available. It is NOT a requirement on SJTAG to provide such information; only if the leveraged standard provides such information. Only the “System Test” knows what information is required to perform the diagnostic task at hand. This is why ATE systems today have various test applications in their arsenal that use an interface differently to obtain the data required to do it’s task (Infrastructure test app, interconnect test app, FPGA Programming app, FLASH programming app, MBIST app, SERDES BERT app, E-MBIST app, CLOCK FREQ Measure app, THERMAL Measure app, BIST app, Board-BIST app, SHELF-BIST app, TRANSMIT IQ Payload Out of Antenna app, STEER RADAR BEAM app, etc.). Depending on the task, the diagnostic knowledge is going to be different. The only thing in common is the infrastructure stimulating and obtaining results from the UUT. Thank you Terry for insisting we concentrate on the behavior of the interface. That will define what kind of information traversing the infrastructure of the UUT. SJTAG needs to be able to handle the data transfer necessary to perform these functions and to preserve important content intent of the leaf entities (this may be an instrument register, device BSR, Board-BIST status register, etc.).
Ian McIntosh
Testability Lead
Leonardo MW Ltd.

User avatar
Ian McIntosh
SJTAG Chair
Posts: 354
Joined: Mon Nov 05, 2007 11:49 pm
Location: Leonardo MW Ltd, UK
Contact:

Re: SJTAG definition of a "system"

Postby Ian McIntosh » Thu Oct 19, 2017 6:45 pm

Transcribing Louis' email of October 17:
Louis wrote:Attached are some ideas for coming up with some definitions of crucial terms for SJTAG. Hopefully we will have time next Monday to discuss these and come up with something we all feel comfortable with.

System
Previous Definition: For the purposes of SJTAG, a system may be described as an organized collection of components or assemblies that are designed to operate together to perform one or more tasks or functions.

Proposed Definition: [based on slides from Prof. K.G. Hambleton, Defence Engineering Group, UCL] - A system is a structured hierarchy of subsystems and components. It has three fundamental properties: a) Boundaries, b) Structure and c) Synergy. Qualifiers: a) Boundaries can never totally isolate the system from its environment, b) Structure comprises a collection of interrelated and interdependent parts often includes a graded hierarchy, c) Synergy implies that the whole is greater than the sum of its parts and that it possesses emergent properties. When boundaries are flexible and change with time, the system may be referred to as a “soft system.”

[From Wikipedia] The Open Systems Interconnection model (OSI model) is a conceptual model that characterizes and standardizes the communication functions of a telecommunication or computing system without regard to their underlying internal structure and technology. Its goal is the interoperability of diverse communication systems with standard protocols. The model partitions a communication system into abstraction layers. The original version of the model defined seven layers:
Layer 1: Physical Layer – Media layer
Layer 2: Data Link Layer– Media layer
Layer 3: Network Layer– Media layer
Layer 4: Transport Layer– Host layer
Layer 5: Session Layer– Host layer
Layer 6: Presentation Layer– Host layer
Layer 7: Application Layer– Host layer

System Test
Previous Definition: None
Related Definitions:
  • Diagnostics: The process of identifying or determining the root cause of a set of test results or observations that are indicative of a malfunction.
  • Diagnosis: The set of conclusions drawn from diagnostics.
  • Diagnostic Tests: Tests intended to determine the cause of a possible malfunction and to suggest a repair strategy.

Proposed Definition: System tests are procedures that test the attributes of a system (as we defined it for this standard). Thus, a system test should be capable of testing the synergistic function of a system, the boundaries and the individual collection of subsystems when they are isolated from other subsystems or parts of the system.

Test Access Mechanism
Previous Definition: The connectivity and protocol structure used to access an instrument. For IEEE Std 1500, a defined TAM exists.

Proposed Definition: The boundary mechanism, connectivity and protocol structure that can be accessed by system test. An important goal of the System Test Access Management (SJTAG) working group will be to describe ways by which access will be available during system test.
Ian McIntosh
Testability Lead
Leonardo MW Ltd.

User avatar
Ian McIntosh
SJTAG Chair
Posts: 354
Joined: Mon Nov 05, 2007 11:49 pm
Location: Leonardo MW Ltd, UK
Contact:

Re: SJTAG definition of a "system"

Postby Ian McIntosh » Sat Oct 28, 2017 7:36 pm

Updated proposals, from Louis' email of Oct 27 (only showing changes from the above):

System
Proposed Definition: - A system is a structured hierarchy of subsystems and components. It has three fundamental properties:
  1. Boundaries, which include not just an interface but also hand-off between the entities it binds
  2. Structure, which is a collection of interrelated and/or interdependent parts that often includes a graded hierarchy, and
  3. Synergy, which implies that the whole is greater than the sum of its parts and that the system possesses emergent properties that are not divisible to its subsystems.

[Illustrative example of synergy: NaCl (salt, a necessity for life) cannot be individually tested for sodium (Na) and chlorine (Cl), both of which are poisonous.]

System Test
Proposed Definition: System tests are procedures that test the properties of a system, including, but not limited to the synergistic properties of a system. For well-structured systems with logically isolated subsystems, the boundaries and the individual subsystems should also be tested sufficiently to determine which subsystem(s) contributed to a system test failure.
Ian McIntosh
Testability Lead
Leonardo MW Ltd.

User avatar
Ian McIntosh
SJTAG Chair
Posts: 354
Joined: Mon Nov 05, 2007 11:49 pm
Location: Leonardo MW Ltd, UK
Contact:

Re: SJTAG definition of a "system"

Postby Ian McIntosh » Mon Oct 30, 2017 2:31 pm

Regarding "System Test": While I'm generally in agreement on the need for a fault isolation capability, I'm not so sure that it's a universal need.

For example, if I want to know if a unit is good to ship (a Final Acceptance Test) or check that a spare unit taken off a shelf is fit to install on an aircraft, I probably only want a Good/Bad indication, and in the event of a "Bad" result I'll just set the item aside. It may be that the diagnostic station is a physically different facility (e.g. with more equipment). I guess I'm thinking that "System Test" is maybe more generic a term than the proposed definition suggests.

Perhaps there ought to be qualified terms and definitions, e.g. "System Diagnostic Test"?
Ian McIntosh
Testability Lead
Leonardo MW Ltd.


Return to “SJTAG General Discussions”

Who is online

Users browsing this forum: Bing [Bot] and 1 guest