Web programming and advanced development techniques Layering LINQ approach XML handling, XLINQ Exercise 1

Web programming and advanced development techniques Layering LINQ approach XML handling, XLINQ Exercise 2

Layering Aim: hiding the implementation details so that the “upper” layer will not depend on other layers not-immediately “below” Similar principle: layered structure of operating systems Az „Alapvető műveletek” listán fogunk mindegyik módszernél végigmenni

Tier vs Layer Layer: logical separation of classes All classes belonging to one layer serve the common “higher purpose” (e.g. display, data management, business logic…) It is clearly defined which classes can be accessed from the outside The external (provided and used) functionalities are described using interfaces We have a well-designed application if a given layer can be replaced with a differently implemented other component that implements the same interface Tier: physical separation of the components of the application Software- or hardware components that execute the layer classes Not necessarily have the same separation as the layers Az „Alapvető műveletek” listán fogunk mindegyik módszernél végigmenni

Application layers Every layer can be separated to internal layers Connecting them: using interfaces Az „Alapvető műveletek” listán fogunk mindegyik módszernél végigmenni

Onion / Hexagonal architecture „Cross cutting concern”, „Aspect oriented programming”

Cross cutting concerns (Aspects) „Cross cutting concern”, „Aspect oriented programming”

Lowest layer: data access/entities We want to implement this layer to be testable & independent from the actual data storage format (db/file, db dialect…) Market Share for databases (Gartner, Mar/2012)

Popularity (db-engines.com, SEP/2018)

Web programming and advanced development techniques Layering LINQ approach XML handling, XLINQ Exercise 10

LINQ (Language Integrated Queries) Allows a simple syntax to handle source-independent management of various collections Simple syntax: simple methods that solve everyday common tasks (that can typically be solved using loops/conditions) Source-independent: array, list, XML, database, … LINQ To Objects, LINQ To XML, LINQ to Entities, ... Parts LINQ operator methods LINQ query syntax (from, where, select, join, orderby...), query parser Lambda expressions Extension methods var keyword, anonymous instances .NET Framework version 3.5-ben jött Forrásfüggetlenségnél az adatbázis után említsd, hogy azért queries, mert a szintaxis eléggé hasonlít a standard SQL szintaxisra. ... = a lánc folytatható, félkövérrel amit ezen az órán veszünk LINQ operátorok = igazából egy csapat függvény LINQ lekérdezések kulcsszavai = where, orderby ... Ezek tartoznak igazából a linq-hoz. A másik három „támogató technológia”  A kiegészítő metódusokat (extension methods) nem tanuljuk

Adding methods ? We want to add new methods to existing collections, BUT … changing the interface definition can have unwanted side effects … we can’t add implemented full methods to interfaces

Extension methods

Lambda expressions with LINQ methods The lambda expressions… Are parameters for the LINQ extension methods Automatically called for every element of the collection If we work with IEnumerable<T>, then the input is T Various possible output types (e.g. WhereFilterbool; OrderBySortIComparable) The lambda outputs are used to determine the output of the LINQ method, which can be IEnumerable<T>, IEnumerable<X>, T, int With databases Same syntax, IQueryable<T> extensions expression lambda  can be merged/optimized!

With extension methods … We have to call the same LINQ method for common tasks, independent from the actual type of the collection The collection is often an interface-typed method or constructor parameter, so the business logic layer doesn’t even know what type it is exactly working with (DEPENDENCY INJECTION, detailed later) The called method is actually an extension method for the interface, that executes the operation (filter/sort/etc) independent from the actual collection type List<T>, XML file  this week Database  next week … … XML format? Soon…

Language constructs Lambda expressions: to shorten the code, and to avoid writing a new method/delegate for every different filter / sort / calculation var type (from C#3): can be only used inside methods; the type of the variable will be deducted by the compiler. We can use it if we do not care about the real type of the variable (The variable is NOT typeless! The variable is NOT dynamic!) We have to instantly assign a value to a var typed variable, and from this point, the usage of the variable is the same (for us and for the Intellisense as well!) var x = 6; var z = new Student();

Language constructs Class initializer: if a class has a zero-parameter constructor: Student joe = new Student() { Credits = 23, Name = "Joe" }; Anonymous classes (when using temporary, small, data-oriented, single-usage classes) : var NameAgeAddress = new { Name="Joe", Age=23, Address="Budapest Bécsi út 96/B" }; Best part: the intellisense will follow this as well! A diák osztálynak most nincs nulla paraméteres konstruktora, a fenti sor csak akkor működik, ha van!

LINQ collections class Student { public string Name { get; set; } public int Credits { get; set; } public Student(string name, int credits) { Name = name; Credits = credits; } public override string ToString() return Name + " - " + Credits; Ezt a példa-osztályt fogjuk használni a példákban

LINQ collections int[] arr1 = new int[] { 2, 4, 6, 8, 2, 1, 2, 3 }; int[] arr2 = new int[] { 1, 3, 5, 7, 1, 1, 2, 3 }; string[] arrStr = new string[] { "Béla", "Jolán", "Bill", "Shakespeare", "Verne", "Jókai" }; List<Student> students = new List<Student>(); students.Add(new Student("Első Egon", 52)); students.Add(new Student("Második Miksa", 97)); students.Add(new Student("Harmadik Huba", 10)); students Add(new Student("Negyedik Néró", 89)); students.Add(new Student("Ötödik Ödön", 69)); Ezeket a gyűjteményeket fogjuk használni a példákban

LINQ Operators Set operators: Concat – concatenation, Contains – check for existance, Distinct – filter repeated values, Intersect – set intersection, Union – set union, Except – set difference Sort: OrderBy – using an arbitrary order, Reverse – reverse order Filter: Where „Calculations” („Aggregate methods”): Average, Count, Max, Min, Sum Grouping: GroupBy – create groups using a common property In the examples, we use arrays/lists Elmondani: csoportosítás működése: valami alapján csoportok képzése + csoportokon ugyanaz a művelet végrehajtása, gyakorlatilag where a csoportképző mező minden egyes értékére, és utána számolás

LINQ Operators 1. – Sets Check for existence of an element: bool isInside=arr1.Contains(4); Concatenate (same elements are not filtered!): var output = arr1.Concat(arr2); Filter out repetitions (convert to set): var output = arr1.Distinct(); Intersection: var output = arr1.Intersect(arr2); Union: var output = arr1.Union(arr2); Difference (elements in arr1 that are not in arr2) var output = arr1.Except(arr2); Elmondani: * Concat() esetén nincsenek kivágva az ismétlődések, csak sima egymás után másolás * halmazelméleti kifejezésnél automatikusan kivágja az ismétlődéseket

LINQ Operators 2. – Reverse In this case, the input of the collection is int[] Even if we filter/reverse  we know the output should be int[] Despite this, we CAN NOT use int[], because the LINQ methods are written as a general approach for all collections  We use IEnumerable<Something> or simply var  Can be converted: result.ToArray(); result.ToList(); var output = arr1.Reverse(); foreach (var akt in output) { Console.Write(akt + ", "); } Console.ReadLine(); 3, 2, 1, 2, 8, 6, 4, 2 Elmondani: IEnumerable<valami> , vagyis a LINQ operátorok kimenetéről általánosságban CSAK annyit tudunk, hogy foreach ciklussal bejárható, a feldolgozást is így végezzük. A foreach ciklus mindig ugyanilyen, a diákon ezentúl a foreach kimenete van csak feltüntetve

LINQ Operators 3. – Sort OrderBy The Lambda expression in the parameter must extract the (IComparable) key from a collection element (which will be used to sort the elements) We could also specify a self-made IComparer Int array, sort the data itself: var output = arr1.OrderBy(x => x); String array, sort according to the length of the elements: var output = arrStr.OrderBy(x => x.Length); List of students, sort by names: var output = students.OrderBy(x => x.Name); Második paraméter: nem fogjuk használni Elmondani: x=>x , Lambda kifejezés: bemenet=kimenet … Az „x” változóval fogjuk jelezni ezentúl az „atuális elem”-et Elmondani: a fordító meg az intellisense is ugyanúgy működik: érzékelik, hogy milyen gyűjteménynek hívom meg a metódusát. string[] esetén x=>x.Length jó, de x=>x.nev nem, ez utóbbi csak diak[] vagy List<diak> esetén jó

LINQ Operators 4. – Filter, Count Where / Count The lambda expression in the parameter must return a bool value The result of the Where is the collection where the lambda returns true. The result of the Count is an int The Count can be called without parameters  full count Int array, odd numbers: var output = arr1.Where(x => x % 2 == 1); String array, number of names with four letters: int num = arrStr.Count(x => x.Length == 4); Második paraméter: nem fogjuk használni Elmondani: x=>x , Lambda kifejezés: bemenet=kimenet Elmondani: a fordító meg az intellisense is ugyanúgy működik: érzékelik, hogy milyen gyűjteménynek hívom meg a metódusát. string[] esetén x=>x.Length jó, de x=>x.nev nem, ez utóbbi csak diak[] vagy List<diak> esetén jó

LINQ Operators 5. – Filter, property selection List of students, where the credits is a prime number: var output = students.Where(x => { if (x.Credits<=1) return false; for (int i = 2; i <= Math.Sqrt(x.Credits); i++) if (x.Credits % i == 0) return false; return true; }); // Második Miksa - 97, Negyedik Néró - 89 Property selection / conversion: var nameList = students.Select(x => x.Name); var jsonList = students.Select(x => x.ToJson());

LINQ Operators 6. – Multiple operators List of students, the uppercase name of the ones with odd credits, sorted by the name, descending: var output = students.Where(x => x.Credits%2==1) .OrderBy(x => x.Name) .Reverse() .Select(x => x.Name.ToUpper()); // or: OrderByDescending // ÖTÖDIK ÖDÖN, NEGYEDIK NÉRÓ, MÁSODIK MIKSA Same result, same IL code (procedural vs declarative syntax): var output = from x in students where x.Credits % 2 == 1 orderby x.Name descending select x.Name.ToUpper(); Elmondani: NEM SQL! Azonos kulcsszavak, mert azonos feladatot látnak el, de más a sorrend, más a használat

LINQ Operators 7. – Aggregation Aggregate methods int mySum = arr1.Sum(); //28 double myAvg = arr2.Average(); //2.875 int sumEven = arr1. Where(x => x % 2 == 0).Sum(); //24 int summOdd = arr1. Where(x => x % 2 == 1).Sum(); //4 The example above happens quite often: we have a Where filter for a flag, and then we execute the same operation for every flag values => execute same calculation for every group One where+calculation per group  slow  automatic calculation: GroupBy

LINQ Operátorok 8. – Group By Group by, according to parity: var myGroup = arr1.GroupBy(x => x % 2); foreach (var g in myGroup) { Console.WriteLine(“Remainder: " + g.Key + ", Number of elements: " + g.Count()); } var output = from x in arr1 group x by x%2 into g select new {Remainder=g.Key, Num=g.Count()}; Mondani: Csoportosításnál jobb a select-es kiírás Az eljáráshalmozás helyett is jobb a select-es kiírás A lambda kifejezések halmozása helyett is jobb a select-es kiírás

Web programming and advanced development techniques Layering LINQ methods XML handling, XLINQ Exercise 29

XML structure (w3schools.com) <?xml version="1.0"?> <bookstore> <book category="COOKING"> <title lang="en">Everyday Italian</title> <author>Giada De Laurentiis</author> <year>2005</year> <price>30.00</price> </book> <book category="WEB"> <title lang="en">Learning XML</title> <author>Erik T. Ray</author> <year>2003</year> <price>39.95</price> </bookstore> XAML elmagyarázása előtt tudni kell, mi az az XML… Hierarchical way to describe data XML declaration + nodes + attributes Nodes/tags: <bookstore></bookstore>, <book></book> = <tag></tag> Attributes: In <book> category=”…”

XML structure Strict rules First line: optional format specification, with character encoding: <?xml version="1.0" encoding="UTF-8"?> Every node can have: Textual value Sub-nodes Attributes One root element must be specified that surrounds the whole document Every tag must be closed (<tag></tag> or <tag />) The nesting of tags must be perfect Bad: <a><b></a></b> Good: <a><b></b></a> Capitalization matters Ezt igazából nem most akarjuk leadni.

XML structure – attribute vs sub-node There is no rule, but use common sense when choosing between attributes or tags. This solution is well-formed, but not logical: <note day="10" month="01" year="2008" to="Tove" from="Jani" heading="Reminder" body="Don't forget me this weekend!"> </note> Suggestion: attributes are quite rare, usually only id="0001", everything else goes into a sub-tag (attributes are hard to extend) Miért? Mert az attribútum nem bővíthető (további alinfókkal), az alelem viszont igen.

XML rules Capitalization matters When defining attributes, quotation or apostrophes are required Use entities for special characters: Karakter Entity < > & & ‘ &apos; " "

XML + .NET XmlReader, XmlWriter Fast, using few memory One-way operation Complicated to create complex XML files XML transformation is HORRIBLE (replacing, exchanging nodes, changing nodes...) XmlDocument, XmlNode, XmlElement, Xml* Slow, uses a lot of memory (builds up the whole XML tree) Algorithms are not necessarily simple (easy for those who are familiar with DOM document operations) Useful for XML transformations XML Serialization Fast, but very limited usage Strict document  class rules

X* classes Aim: Fast Clean design Support the standard XML tools (XQuery, XSLT) Replace XPath XDocument, XElement, XAttribute, XDeclaration, X* Very easy to create/read XML files in .NET System.Xml.Linq namespace Automatic string conversion (with tag names!) We have to typecast after reading the Value property

XElement The constructor is flexible (params keyword)  almost any XML can be created with a single constructor call var xe = new XElement("person", "Joe"); <person>Joe</person> var xe2 = new XElement("person", new XElement("name", "Joe"), new XElement("age", 25)); <person> <name>Joe</name> <age>25</age> </person> Params  el kell magyarázni

XAttribute With constructor: var xe = new XElement("person", new XAttribute("id", 43984), new XElement("name", "Joe"), new XElement("age", 25)); After the XElement creation: var xe2 = new XElement("person", xe2.SetAttributeValue("id", 43984); <person id="43984"> <name>Joe</name> <age>25</age> </person>

To save an XDocument XDocument outDoc = new XDocument( new XElement("people", new XElement("person", new XAttribute("id", 0), new XElement("name", "Joe"), new XElement("age", 22)), new XAttribute("id", 1), new XElement("name", "Quagmire"), new XElement("age", 34)))); outDoc.Save("people.xml"); Customizable: outDoc.WriteTo(…) + XmlWriter

Simple data processing <people> <person id="43984"> <name>Joe</name> <age>25</age> <phone>0618515133</phone> </person> … </people> .Element(name) .Attribute(name) .Elements(name) .Attributes(name) string name = xDoc.Element("people").Element("human") .Element("name").Value; int id = int.Parse(xDoc.Element("people").Element("human") .Attribute("id").Value); ... OR ... (typecasting works with attributes and elements too) int id = (int)(xDoc.Element("people").Element("human") .Attribute("id"));

LINQ to XML X* classes: strong LINQ-support! A lot of methods return IEnumerable<T>, T is usually XElement / XAttribute Pl. XElement xe2: xe2.Descendants() Every descendants (including sub-descendants) xe2.Descendants("note") Every descendants with a given tag name (including sub-descendants) xe2.Elements() Immediate descendants xe2.Elements("note") Immediate descendants with a given tag name xe2.Attributes(), xe2.Ancestors() … Descendants()  paraméterezés nélkül minden gyermekelemen (a gyermekelem gyermekelemein is!) végigmegy

LINQ to XML <people> <person id="43984"> <name>Joe</name> <age>25</age> <phone>0618515133</phone> </person> … </people> XDocument XDoc = ... var q = from x in XDoc.Root.Descendants("person") where x.Element("name").Value.StartsWith("Jo") select x;

Web programming and advanced development techniques Layering LINQ methods XML handling, XLINQ Exercise 42

Load XML XDocument inDoc = XDocument.Parse(stringWithFullXML); XDocument inDoc = XDocument.Load("Joe.xml"); XDocument XDoc = XDocument.Load ("http://users.nik.uni-obuda.hu/prog3/_data/people.xml"); A „random” értéke pszeudo-random 0-255: a név bináris MD5-jének második byteja decimálisan

Example List those who do not work in the building BA (We work with String data – we always have to typecast to the good type if this is not good)! Extra keyword: LET – to define local variables for the LINQ query XDocument XDoc = XDocument.Load("http://users.nik.uni-obuda.hu/prog3/_data/people.xml"); var q0 = from akt in XDoc.Descendants("person") let room=akt.Element("room").Value where !room.StartsWith("BA") select akt; foreach (var akt in q0) { Console.WriteLine(akt.ToString()); }

Exercise The XElement class can be processed using the .Element() and .Attribute() methods... Using those, import the XML into a list! Determine the number of workers who work in the AII department! Display those who live in the third floor in a “paginated” list, we should press ENTER after every 15 elements Who have the shortest or the longest names? Determine the number of workers for every department! Determine the biggest department! List the workers for the biggest department! List the people who work in the third biggest department ordered by their room number in descending order!

2nd Exercise http://users.nik.uni-obuda.hu/prog3/_data/war_of_westeros.xml Ezen fogunk dolgozni

2nd Exercise In the war of five kings … How many houses participated? List the battles with the „ambush” type! How many battles are there where the defending army won and there was a major capture? How many battles were won by the Stark house? Which battles had more than 2 participating houses? Which are the 3 most violent regions? Which one is the most violent region? In the 3 most violent region, which battles had more than 2 participating houses? (Q5 join Q6) List the houses ordered descending by the number of battles won! Which battle had the biggest known army? List the three commanders who attacked the most often!

Sources Lambda expressions: http://msdn.microsoft.com/en-us/library/bb397687.aspx Lambda expressions: http://geekswithblogs.net/michelotti/archive/2007/08/15/114702.aspx Why use Lambda expressions: http://stackoverflow.com/questions/167343/c-lambda-expression-why-should-i-use-this Recursive lambda expressions: http://blogs.msdn.com/b/madst/archive/2007/05/11/recursive-lambda-expressions.aspx Standard query operators: http://msdn.microsoft.com/en-us/library/bb738551.aspx Linq introduction: http://msdn.microsoft.com/library/bb308959.aspx 101 Linq samples: http://msdn.microsoft.com/en-us/vcsharp/aa336746 Lambda: Reiter István: C# jegyzet (http://devportal.hu/content/CSharpjegyzet.aspx) , 186-187. oldal Linq: Reiter István: C# jegyzet (http://devportal.hu/content/CSharpjegyzet.aspx) , 250-269. oldal Fülöp Dávid XLinq prezentációja Linq to XML in 5 minutes: http://www.hookedonlinq.com/LINQtoXML5MinuteOverview.ashx Access XML data using Linq: http://www.techrepublic.com/blog/programming-and-development/access-xml-data-using-linq-to-xml/594 Simple XML parsing examples: http://omegacoder.com/?p=254 , http://gnaresh.wordpress.com/2010/04/08/linq-using-xdocument/ XML: Reiter István: C# jegyzet (http://devportal.hu/content/CSharpjegyzet.aspx) , 224. oldal (A könyv az XMLReader/Writer, illetve az XmlDocument használatát mutatja be)