Q: Basic question regarding n of 1 study. Four weeks intervention I initially will apologise for any lack of clarity with regards to this question. I'm posing this on behalf of my sibling who is in the biomedical sciences.
They have done an intervention on a patient. They are measuring the effect of the intervention by looking at test scores (6 questions), and repeating this test over four weeks.
The data looks like this
Week 1: 4/6
Week 2: 5/6
Week 3: 5/6
Week 4: 6/6
My sibling wants to see if the increase in the test scores is significant or not. I didn't really know how to go about this.
Any help would be greatly appreciated

A: Firstly, this is a very small sample size (4 observations), hopefully you have more, or at least have the same four observations for many different patients. If not, it will be difficult to find a model. Generally it is good to have a sample size greater than 100, or at a bare minimum, 20.
Secondly, the survey (6 questions) is also small. Does the patient receive the same questions each time? If so, they can be more prepared next time. But, this doesn't answer your question, but tell your sibling to seriously consider the set-up of the survey.
That being said, if this is the only data you have, there are a couple quick hacks. First, just regress the survey data on the index (you can make it just 1:4 as long as the surveys were taken at regular intervals):
> summary(lm(c(4,5,5,6)~c(1,2,3,4)))

Call:
lm(formula = c(4, 5, 5, 6) ~ c(1, 2, 3, 4))

Residuals:
   1    2    3    4
-0.1  0.3 -0.3  0.1

Coefficients:
              Estimate Std. Error t value Pr(>|t|)
(Intercept)     3.5000     0.3873   9.037   0.0120 *
c(1, 2, 3, 4)   0.6000     0.1414   4.243   0.0513 .
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.3162 on 2 degrees of freedom
Multiple R-squared:   0.9,  Adjusted R-squared:  0.85
F-statistic:    18 on 1 and 2 DF,  p-value: 0.05132

Here we see that the slope has a p-value of .0513, making it significant with 94.87% confidence. On average, at each step, the patient improves by .6 points. It seems obvious that there is a significant increase over time based on the sample size. However, the next question would be why does the increase occur?

A: Caveat emptor: I am NOT a biostatistician...but I play one on TV ;-) In all seriousness, I have a strong statistics background, but it is not medically oriented. However, your question was simply stated, so my suggestions utilize a general approach, not one that is domain specific to biomedical research.
First, you have a very small sample size. I hope your sibling isn't looking for anything too exact! Also, I am assuming the test your sibling is using has been properly validated and calibrated, and that your sibling has access to that data. The validation/calibration data will add MUCH to this analysis. Without it, you have limited power. I am also surprised that a biomedical researcher doesn't have ready access to a trained biostatistician. However, I digress....
What you have presented is a short time series. What your sibling would like to know is if there is evidence for a positive trend. Without any assumptions about how these scores are supposed to behave, I can propose a simple resampling approach that may help shed light on the strength of the evidence:
If there were no trend, then the above scores can be seen as 4 samples from the same population, as there would be no difference between week 1 vs week 2 etc.
There are two approaches to quantifying the trend that I see:

*

*There is a net change of 2 units in 4 weeks?

*The scores increased twice, and never decreased.

The difference between the two is that (1) cares only about the overall change, which would allow a situation where there is no change for 3 weeks then rapid improvement in the last week while (2) cares more about the consistency of the behavior vs the magnitude of the effect. I'll outline both, but given the small range of possible scores, (2) may be more appropriate.
Resampling Approach
The null hypothesis is the same in both cases: $H_0$: the samples were drawn from the same distribution vs $H_a$: The samples were drawn from distributions with increasing medians.
Since we are assuming for $H_0$ that all samples are from the same distribution, you should draw a sample of size 4, with replacement, from the set of scores {4,5,5,6} and assign each score to a week: $W_1,W_2,W_3,W_4$. Then, calculate the relevant measure of trend (i.e., 1 or 2 above) and record its value. Repeat this a large number of times (say 1000) and see how often you get a trend value at least as big as the one for your sample. This will indicate how likely it would have been to see this type of improvement even if there were no real improvement. Obviously, you will need a computer stats package, like R or Minitab, that allows resampling, to do this effectively.
Specifically:
For 1: The measure of trend, T, is $T=W_4 - W_1$. You will want to know how many of your resampling trials result in $T=2$.
For 2: The measure of trend is T= (# of inter-week increases) - (# of inter-week decreases). For example if one of the resamples give us {6,4,5,6} we get T = 2 - 1 = 1.
I doubt this method will give very strong results either way, given the small sample size. If your sibling has some prior information about the distribution of test scores for healthy individuals vs sick individuals, then you could do A LOT more, since you will know the typical range and score for each. For example, if healthy patients score 5 30% of the time and 6 70% of the time while a sick patient has score distribution {1 (10%), 2(40%), 3(30%), 4(15%), 5(5%), 6(0%)} Then the above scores carry much more weight.
Context matters, so without any, you have very limited power. With more context, the tests could be quite powerful..that is the value of properly validated and calibrated tests.
Numerical Results

Distribution of differences between Week 4 and Week 1

Distribution of net inter-week direction changes
Surprisingly to me, the observed test statistics is are quite unusual if the patient were showing now improvment and the test scores were all noise.

A: How about this: Take the null model that at each timepoint there is 1/3 chance the score will be higher, lower, or the same as the previous timepoint. The probability you would get no scores worse than the previous week 3 weeks in a row is (2/3)X(2/3)X(2/3) = 8/27 = 0.296
Edit:
Here is one way to look at it using an approach similar to that proposed by @Eupraxis1981. Lets say the intervention does nothing, the test measures nothing but noise, and the first score on the test=4. This histogram shows distribution of slopes could we expect:

Under these conditions the slope would be greater than or equal to the observed slope of 0.6 about 12% of the time. I still think there should be a way to rule out large negative effects of the treatment though.
R code:
x<-c(1,2,3,4)
y<-c(4,5,5,6)
actual.slope<-round(lm(y~x)$coefficients[2],2)

probs<-rep(1/6,6)
n=100000
sim.slopes<-matrix(nrow=n,ncol=1)
for(i in 1:n){
  tmp<-c(4,sample(1:6,3, replace=T, prob=probs))

  sim.slopes[i,]<-round(lm(tmp~x)$coefficients[2],2)
}

perc<-round(100*length(sim.slopes[which(sim.slopes>=actual.slope)])/length(sim.slopes),2)

hist(sim.slopes,breaks=seq(-2,2,by=.1),
     xlab="Slope", col="Grey",
     main=paste(perc,"% of slopes >=", actual.slope ))
hist(sim.slopes[which(sim.slopes>=actual.slope)],
     breaks=seq(-2,2,by=.1), add=T, col="Red")


Q: Protobuf generated file doesn't recognize import path I have a proto file product.proto and I want to use it in catalog.proto. I can successfully import product.proto in catalog.proto but auto-generated catalog.pb.go is not able to reconize the path of its dependency i.e. product.pb.go. It states:

could not import catalog/pb/entities (cannot find package "catalog/pb/entities" in any of
/usr/local/go/src/catalog/pb/entities (from $GOROOT)
/Users/trimulabs/go/src/catalog/pb/entities (from $GOPATH))

Directory structure
catalog
 ┣ main
 ┃ ┣ client.go
 ┃ ┗ server.go
 ┣ pb
 ┃ ┣ entities
 ┃ ┃ ┣ product.pb.go
 ┃ ┃ ┗ product.proto
 ┃ ┣ catalog.pb.go
 ┃ ┗ catalog.proto

catalog.proto
syntax = "proto3";
package catalog;
import "catalog/pb/entities/product.proto";

service CatalogService {
    rpc PostProduct (PostProductRequest) returns (PostProductResponse) {}
    rpc GetProduct (GetProductRequest) returns (GetProductResponse) {}
    rpc GetProducts (GetProductsRequest) returns (GetProductsResponse) {}
}

product.proto
message Product {
    string id = 1;
    string name = 2;
    string description = 3;
    double price = 4;
}
message PostProductRequest {
    string name = 1;
    string description = 2;
    double price = 3;
}
message PostProductResponse {
    Product product = 1;
}

message GetProductRequest {
    string id = 1;
}

message GetProductResponse {
    Product product = 1;
}

message GetProductsRequest {
    uint64 skip = 1;
    uint64 take = 2;
    repeated string ids = 3;
    string query = 4;
}

message GetProductsResponse {
    repeated Product products = 1;
}

Can anyone please help?

A: The error you are seeing is caused because your generated protobuf file likely has an incorrect import path.
If you check your product.pb.go file, it likely has a like like:
import "catalog/pb/entities"

This means it is trying to import the go package with that specified path. As you are probably familiar with, a go package must use the absolute path, so this needs to look something like:
import "github.com/<myuser>/<myproject>/catalog/pb/entities"

protoc-gen-go fortunately supports a couple of flags to help you with this.
Step 1. Set go_package in your .proto file to specify the anticipated import path in go.
option go_package = "example.com/foo/bar";

Step 2: use the --go_opt=module=github.com/<myuser>/<project> flag of protoc to strip the module prefix from your generated code. Otherwise you would get something like this as an output (assuming you are using the current directory as your go_out path
catalog/
  main/
  pb/
  github.com/
    <myuser>/
      <project>/
        catalog/
          pb/
            entities/

I would recommend reviewing the documentation page on the protoc-gen-go go flags:


Q: What are my custom reply-to addresses not used in MC? I have created a Sender Profile with custom reply-to name/address fields defined. When doing a send preview you can clearly see the values differ from the From Name/Address fields however when I receive the email and try to respond, it responds to the From Address.
It should be noted I've disabled RMM, by unchecking "Has DNS been updated".
Can anyone explain why my email sends don't have the correct reply-to addresses?
Edit: Added screenshots to show what settings have been used, redacted part of the domain and sender names.

A: I actually wanted to see the bottom part of your Sender Profile, but let's give it a shot - try configuring the bottom part of the Sender Profile in the following way:


Q: Prove that $a|p+1$ if and only if ${\frac{a}{p} = \frac{1}{n} + \frac{1}{m} }$ such that $n,m \in N$ and $p$ is prime Let $p$ be a prime and let $n$ be a natural number such that $n \gt a$ .
Prove that $a|p+1$ if and only if exists integers $n,m$
Such  that    ${\frac{a}{p} = \frac{1}{n} + \frac{1}{m}}$

A: One direction:
Say $${a\over p}= {1\over m}+{1\over n}\implies amn = p(m+n)$$
Let $g= \gcd(m,n)$ then exists $x,y$ relatively prime such that $m= gx$ and $n= gy$. Then we have $$axyg = p(x+y)\implies x\mid p(x+y) \implies x\mid py \implies x\mid p $$
Case 1: $x=p$ then $ayg = p+y \implies y\mid p\implies y= 1 \implies a\mid p+1$
Case 2: $x=1$ then $ayg = p(1+y) \implies y=1\; \text {or}\; y=p$
If $y=1$ then $m=n$ and $2p= an$ then we can take $n=p$ and $a=2$ and the statement is not true!
If $y=p$ then we get $a\mid p+1$ and we are done.
Other direction: $a\mid p+1$ then $p+1 =an$ so ${a\over p} = {1\over p} +{1\over pn}$ and we are done.

A: $a\mid p+1 \iff a\cdot n=p+1\\ \iff\frac{a\cdot n}{p}=1+\frac{1}{p} \\ \iff\frac{a}{p}=\frac{1}{n}+\frac{1}{n\cdot p}$
Let $m:=n\cdot p$ and we are done


Q: How to decide between quasi-poisson and negative binomial? I tried quasi-poisson and negative binomial glms on my counted data in R. The estimates are pretty much the same but p-value are different. Quasi-poisson gives insignificant result. NB give significant result. I am not sure how to decide between the two models. I computed QAIC for quasi but I read a paper saying that it is created to compare within quasi family. Another one says we should compute to compare with AIC. Is there any other way to compare the two models?
                        Quasi-poisson   NB
Deviance/DF              994.93       3.0014

Person chi-square test   995.64       2.997

QAIC/AIC                 5.041        68.47


Q: Enter the passcode you use to unlock the iPad - what is it? I am trying to connect to iTunes from my MacBook and it asks me "the passcode you use to unlock the iPad" and naminf my iPad.
Questions:

*

*Does this mean that pin code pin code, which I use on iPad?

*How can MacBook know it? Is it sended to Apple?

*I am using Russian letters to unlock iPad, but on MacBook it is not possible to switch to Russian input. How can I enter unlock code then?

I have changed my unlock code to English letters, but this didn't help. Why do they need all moy codes?!

A: *

*it is the pin code to unlock the iPad

*MacBook only "knows" that the iPad has been unlocked. Pin is not sent to Apple.

Not necessary for connecting iPad and MacBook, but anyway: if you want to install Russian letters on MacBook, go to SystemPreferences > Keyboard > Input Sources (4th Tab from left side). Hit the [+] Button below left column and choose Russian.


Q: Defining a Generic Method in a Trait When Using Algebraic Data Type in Scala I have defined an abstract data type, and i wanted to make it fairly general and scalable.
However, scala type system is giving me a nightmare.
sealed trait Tree[+A] {
  def evaluateTree(): A = this match {
    case Leaf(value) => value
    case Branch_A1(op, left) => op(evaluateTree(left))
    case Branch_A2(op, left, right) => op(evaluateTree(left),evaluateTree(right))
  }
}
case object EmptyTree extends Tree[Nothing]
case class Leaf[A](value: A) extends Tree[A]
case class Branch_A1[A](op: A => A, left: Tree[A]) extends Tree[A]
case class Branch_A2[A](op: (A,A) => A, left: Tree[A], right: Tree[A]) extends Tree[A]

A would be e.g. Double. So in this case i have a tree which has branches that represent functions of one and two arguments (Branch_A1, Branch_A2)
However it does not compile. In op(evaluateTree(left)) it says that "it cannot resolve ... with such reference".
I could take the function away from the class into a more functional pattern but i want to do it following an object design. The only way i can get it into compiling is to do:
sealed trait Tree[+A] {
  def evaluateTree(t: Tree[A]): A = this match {
    case Leaf(value) => value
    case Branch_A1(op, left) => op(evaluateTree(left))
    case Branch_A2(op, left, right) => op(evaluateTree(left),evaluateTree(right))
  }
}
case object EmptyTree extends Tree[Nothing]
case class Leaf[A](value: A) extends Tree[A]
case class Branch_A1[A](op: A => A, left: Tree[A]) extends Tree[A]
case class Branch_A2[A](op: (A,A) => A, left: Tree[A], right: Tree[A]) extends Tree[A]

which is stupid since i need to pass it the instance of the object.
I need to tell the compiler that this is an Tree[A]. So i have tried:
sealed trait Tree[+A] {

  this: Tree[A] =>

  def evaluateTree(): A = this match {
    case Leaf(value) => value
    case Branch_A1(op, left) => op(evaluateTree(left))
    case Branch_A2(op, left, right) => op(evaluateTree(left),evaluateTree(right))
  }
}

Still the compiler does not let me go, the error is the same actually.
How can i solve this??

A: It looks like your receivers are screwed up. evaluateTree does not take arguments. I'm assuming you want to evaluate the subtree, as in op(evaluateTree(left)) should instead be op(left.evaluateTree())
def evaluateTree(): A = this match {
  case Leaf(value) => value
  case Branch_A1(op, left) => op(left.evaluateTree())
  case Branch_A2(op, left, right) => op(left.evaluateTree(),right.evaluateTree())
}


Q: Rails Serializer Association I have a Component class that can be either a Section or Question. A Section can have many Component (i.e. Question).
I'm trying to get the serializer to send back the Component(Section) and its associated Component(Question). Right now, based on my code below, I'm getting back the Question and the Section object that it's associated to (see below). How can I get the serializer to return what I'm expecting?
Response (current):
{"data":
  [{"id": 1,
    "type": "Section",
    "content": "ABC",
    "section_id": null,
    "section": null
   },
   {"id": 2,
    "type": "Question",
    "content": "Q1",
    "section_id": 1,
    "section":
      {"id": 1,
       "type": "Section",
       "content": "ABC",
       "section_id": null
      }
   },
   {"id": 3,
    "type": "Question",
    "content": "Q2",
    "section_id": 1,
    "section":
      {"id": 1,
       "type": "Section",
       "content": "ABC",
       "section_id": null
      }
   }]
}

Response (expected):
{"data":
  [{"id": 1,
    "type": "Section",
    "content": "ABC",
    "section_id": null,
    "section":
       {"id": 2,
        "type": "Question",
        "content": "Q1",
        "section_id": 1
       },
       {"id": 3,
        "type": "Question",
        "content": "Q2",
        "section_id": 1
       }
   }]
}

ComponentSerializer:
class ComponentSerializer < ActiveModel::Serializer

  belongs_to :section

  attributes :id,
         :type,
         :content,
         :section_id
end

SectionSerializer:
class ComponentSerializer < ActiveModel::Serializer

  has_many :components

  attributes :id,
         :type,
         :content,
         :section_id
end

component.rb (model):
class Component < ApplicationRecord

  # == Associations ==========================
  belongs_to :project
  belongs_to :section,
    class_name: 'Section',
    foreign_key: :section_id,
    optional: true

end

section.rb (model):
class Section < Component

  # == Associations ==========================
  has_many :components,
    class_name: 'Component',
    foreign_key: :component_id,
    dependent: :destroy

end

component_controller.rb:
  before_action :load_project
  before_action :load_scope

  def index
    components = paginate @scope, per_page: per_page
    data = ActiveModelSerializers::SerializableResource.new(
      components,
      each_serializer: ComponentSerializer
    )
    render_response(:ok, { data: data })
  rescue ActiveRecord::RecordNotFound
    render_response(:not_found, { resource: "Project/Lesson" })
  end

  def load_scope
    @scope = @project.components
    @scope = @scope.order("position")
  end

  def load_project
    @project = Project.find(params[:project_id])
  rescue ActiveRecord::RecordNotFound
    render_response(:not_found, { resource: "Project/Lesson" })
  end


Q: Python module ecdsa errors while running paramiko I am trying to install paramiko module..it fails with the below error
python ./test.py
Traceback (most recent call last):
  File "./test.py", line 30, in <module>
    import paramiko
  File "/tmp/build-paramiko/paramiko-1.12.0/paramiko/__init__.py", line 64, in <module>
    from transport import SecurityOptions, Transport
  File "/tmp/build-paramiko/paramiko-1.12.0/paramiko/transport.py", line 45, in <module>
    from paramiko.ecdsakey import ECDSAKey
  File "/tmp/build-paramiko/paramiko-1.12.0/paramiko/ecdsakey.py", line 24, in <module>
    from ecdsa import SigningKey, VerifyingKey, der, curves
ImportError: No module named ecdsa

Any suggestions on how to proceed with the paramiko installation ?

A: Download the package from 'https://github.com/warner/python-ecdsa' and install it using command
python setup.py install

Your problem will be solved.

A: You can use easy_install to install the lost module "ecdsa" ,which like: easy_install ecdsa,
but you have to ready easy_install first!

A: this:
    from ecdsa import SigningKey, VerifyingKey, der, curves
ImportError: No module named ecdsa

suggests that the python-ecdsa package is missing, you can install it with
pip install ecdsa

Though in general, you shouldn't need to install paramiko from sources. You can install it with
pip install paramiko

that has the benefit of automatically resolving the dependencies of a package


Q: Visibility in a prime orchard This suggests a variant on Polya's orchard problem.
That problem asks1
for which radius $\epsilon$ of trees at each lattice point within a distance $R$ of the origin block all lines of sight to the exterior of the orchard.
It was established in the 1980's that rays to infinity are completely blocked iff
$\epsilon \ge  1/\sqrt{R^2 + 1}$, when $R$ is an integer.2
E.g., for $R=20$, $\epsilon = 1/\sqrt{401} \approx 0.05$ suffices.
The variant is: Only lattice points both of whose coordinates
are prime numbers are fattened to $\epsilon$-trees,
and the source of visibility is $(2,2)$ rather than $(0,0)$.
For example, the below shows that $\epsilon=0.3$ is insufficient
to block visibility for $R=20$:

The basic question is:

Q1.
  For a given $R$, which $\epsilon$ suffices to block visibility
  from $(2,2)$ to beyond $R$ ?

As this does not seem easy to answer precisely, let me pose a very specific
question:

Q2.
  As $R \to \infty$, does $\epsilon \to 0$?

It could be that the thinning of the prime trees
is rapid enough to require some lower bound on $\epsilon$
to block visibility, even for large $R$.

1Here I am
quoting from a previous MO question.

2Thomas T. Allen, "Polya's orchard problem."
(Jstor link.)
The American Mathematical Monthly
93(2): 98-104 (1986).)


Q: List specific files in command that expects `ls` piped into it I'm looking for an elegant cross-compatible way to direct the names of files into entr. I know the following works with Bash, but I'd like a command that would work in other shells.
(entr is a program that watches files and runs a command on start and when those files change. Its documentation shows ls *.ext, for example, piped into it - so it is expecting a newline separated list of legitimate filenames piped to it. It warns on bad filenames.)
multiliners
This command lets me list specific files with a multiline herestring:
$ entr echo "command ran" <<< "
> filename1
> filename2
> "

I find the quotation marks and having to list them on separate lines to be annoying. Also a quick search indicates it might not work in shells other than Bash.
I could use interactive cat:
$ cat | entr echo "command ran"
filename2
filename2

but again I have to do a newline each time, not much margin for error, and carefully enter Ctrl-D at the end.
A heredoc works similarly but involves bookend syntax.
$ cat << EOF | entr echo "command ran"
> filename1
> filename2
> EOF

one-liners:
I could also do ls, which looks nice at first glance:
$ ls filename1 filename2 | entr echo "command ran"

But ls would strip out misspelled names, while entr validates and warns on improper names, and I'm worried about side-effects of calling ls in this way.
I could echo and pipe to tr before piping to entr:
$ echo filename1 filename2 | tr " " "\n" | entr echo "command ran"

But I don't like having all the extra commands.
The Question:
How do I write a short, elegant command that works in most (preferably all) shells? Syntax expected by posix would be ideal.

A: Since entr appears to want each file on a separate line you're pretty limited.  You have to have each filename on a separate line.  Any attempt to make it a single line (eg your tr example) will fail if the separator character is part of a filename ("this is a filename").
I'd just do the simple
echo 'filename 1
filename2' | entr echo "command ran"


Q: Select only 1 payment from a table with customers with multiple payments I have a table called "payments" where I store all the payments of my costumers and I need to do a select to calculate the non-payment rate in a given month.
The costumers can have multiples payments in that month, but I should count him only once: 1 if any of the payments is done and 0 if any of the payment was made.
Example:
+----+------------+--------+
| ID |  DATEDUE   | AMOUNT |
+----+------------+--------+
|  1 | 2016-11-01 | 0      |
|  1 | 2016-11-15 | 20.00  |
|  2 | 2016-11-10 | 0      |
+----+------------+--------+

The result I expect is from the rate of november:
+----+------------+--------+
| ID |  DATEDUE   | AMOUNT |
+----+------------+--------+
|  1 | 2016-11-15 | 20.00  |
|  2 | 2016-11-10 | 0      |
+----+------------+--------+

So the rate will be 50%.
But if the select is:
SELECT * FROM payment WHERE DATEDUE BETWEEN '2016-11-01' AND '2016-11-30'

It will return me 3 rows and the rate will be 66%, witch is wrong. Ideas?
PS: This is a simpler example of the real table. The real query have a lot of columns, subselects, etc.

A: Try this
SELECT
    id
    , SUM(AMOUNT) AS AMOUNT
FROM
    Payment
GROUP BY
    id;

This might help if you want other columns.
WITH cte (
    SELECT
        id
        , ROW_NUMBER() OVER (PARTITION BY ID ORDER BY AMOUNT DESC ) AS RowNum
        -- other row
)
SELECT *
FROM
    cte
WHERE
    RowNum = 1;

A: It sounds like you need to partition your results per customer.
SELECT        TOP 1 WITH TIES

              ID,
              DATEDUE,
              AMOUNT

ORDER BY      ROW_NUMBER() OVER (PARTITION BY ID ORDER BY AMOUNT DESC)

WHERE DATEDUE BETWEEN '2016-11-01' AND '2016-11-30'

PS: The BETWEEN operator is frowned upon by some people.  For clarity it might be better to avoid it:

*

*What do BETWEEN and the devil have in common?

A: To calculate the rate, you can use explicit division:
select 1 - count(distinct case when amount > 0 then id end) / count(*)
from payment
where . . .;

Or, in a way that is perhaps easier to follow:
select avg(flag * 1.0)
from (select id, (case when max(amount) > 0 then 0 else 1 end) as flag
      from payment
      where . . .
      group by id
     ) i


Q: Apache Camel aggregator in combination with onCompletion I want the onCompletion to occur after all the aggregated exchanges triggered by both completion size followed by timeout are processed. But it occurs right after the completion size is triggered with some of the exchanges waiting to be triggered by the timeout criteria.

A: I have the route configured as
from(fromEndPoint)
    .onCompletion()
    .doSomething()
    .split() // each Line
    .streaming()
    .parallelProcessing()
    .unmarshal().bindy
    .aggregate()
    .completionSize(100)
    .completionTimeout(5000)
    .to(toEndpoint)
Assume if the split was done on 405 lines, the first 4 sets of aggregated exchanges go to the to endpoint completing 400 lines(exchanges) . And then, it immediately triggers the onCompletion. But there are still 5 more aggregated exchanges which would be triggered when the completionTimeout criteria is met. It didn't trigger the onCompletion after the 5 exchanges are routed to the to endpoint.
My question here is , either the onCompletion should be triggered for each exchange or once after all.
Note:- My from endpoint here is a File.


Q: split cell at special character if comma found after first word hi i've got some budget data with names and titles that read "Last, First - Title" and other rows in same column position that read "anything really - ,asd;flkajsd". I'd like to split the column IF first word ends in a "," at the "-" position that follows it.
ive tried this:
C22$ITEM2 <- ifelse(grepl(",", C22$ITEM), C22$ITEM,  NA)
test <- str_split_fixed(C22$ITEM2, "-", 2)
C22 <- cbind(C22, test)

but i'm getting other cells with commas elsewhere, need to limit to just "if first word ends in comma"

A: library(tidyverse)

data <- tibble(data = c("Doe, John - Mr", "Anna, Anna - Ms", " ,asd;flkajsd"))
data
data %>%
  # first word must ed with a
  filter(data %>% str_detect("^[A-z]+a")) %>%
  separate(data, into = c("Last", "First", "Title"), sep = "[,-]") %>%
  mutate_all(str_trim)
# A tibble: 1 × 3
#  Last  First Title
#  <chr> <chr> <chr>
#1 Anna  Anna  Ms

A: We may use extract to do this - capture the regex pattern as two groups ((...)) where the first group would return word (\\w+) from the start (^) of the string followed by a ,, zero or more space (\\s*), another word (\\w+), then the - (preceding or succeeding zero or more space and the second capture group with the word (\\w+) before the end ($) of the string
library(tidyr)
library(dplyr)
extract(C22, ITEM, into = c("Name", "Title"),
      "^(\\w+,\\s*\\w+)\\s*-\\s*(\\w+)$") %>%
   mutate(Name = coalesce(Name, ITEM), .keep = 'unused')

NOTE: The mutate is added in case the regex didn't match and return NA elements, we coalesce with the original column to return the value that corresponds to NA


Q: BEM: What are modifiers allowed to modify? Let's say I have the following CSS for a generic list component, using BEM and SCSS:
.list {
  &__item {
    &:not(:last-child) {
      margin-right: .3em;
    }
  }
}

I want to add a modifier that can make the list vertical, like so:
.list--vertical {
  display: flex;
  flex-direction: column;
}

My problem arises when I think about the margin for list__item elements. For vertical lists, I want my margin on the bottom, not right of each item. If I understood BEM correctly, I cannot modify the style of list__item based on the modifer of list, is that correct?
To be more precise, this is what I want to do:
.list--vertical {
  display: flex;
  flex-direction: column;

  .list__item {
    &:not(:last-child) {
      margin-bottom: .3em;
      margin-right: 0;
    }
  }
}

What is the accepted way of dealing with this using BEM? Another modifier for list__item that handles the margin? Another block entirely for vertical lists?

A:
What is the accepted way of dealing with this using BEM?

Depends on what version of BEM you're using. I use a variant of the pre-spec concept of BEM, which means that you'll have different answers if you follow bem.info.

Modifiers should be attached to the element they modify. Modifying a block, however, allows the modifier to be inherited by child elements:

...

This gets messy when child elements have modifiers as well:

...

This form of BEM syntax is quite verbose. It's best used on generated code.

I use LESS for CSS preprocessing, so my BEM code often looks like:
.foo {
  ...

  &__bar {
    ...
  }
}

With modifiers it becomes:
.foo {
  ...

  &__bar {
    ...
  }

  &--example {
    ...

    &__bar {
      ...
    }
  }
}

This enforces that the cascade is in the proper order and that all selectors continue to have exactly one class.


Q: Alternating Series Test proof (induction) Alternating Series Test
Let $(a_n)$ be a sequence satisfying,

*

*$a_1 \geq a_2 \geq a_3 \geq \cdots \geq a_n \geq a_{n+1} \geq \cdots  $

*$(a_n) \rightarrow 0$
Then, the alternating series $\sum_{n=1}^{\infty} (-1)^{n+1} a_n$ converges.

In the book I'm reading the proof tries to show that $s_n = \sum_{k=1}^{n} (-1)^{k+1} a_k$ is a Cauchy sequence. So it's trying to prove that given $\epsilon > 0$ there is a $N$ that when $n, m > N$ then:
$|a_{m+1} - a_{m+2} + \cdots \pm a_{n}| < \epsilon$
This would imply it converges by the Cauchy Criterion.
The part I don't understand is when it says that by an induction argument it is possible to show that:
$|a_{m+1} - a_{m+2} + \cdots \pm a_{n}| < |a_{m+1}|$
Base case is clear.
Now assume that:
$|a_{m+1} - a_{m+2} + \cdots \pm a_{n}| < |a_{m+1}|$
We want to prove that:
$|a_{m+1} - a_{m+2} + \cdots \pm a_{n+1}| < |a_{m+1}|$
I could say $|(a_{m+1} - a_{m+2} + \cdots \pm a_{n}) \pm a_{n+1}| \leq |a_{m+1} - a_{m+2} + \cdots \pm a_{n}| + |a_{n+1}| < |a_{m+1}| + |a_{n +1}|$
But this is not the result I'm looking for.
I could try with the terms $a_{n+2},\ldots,a_n$ but can't justify my argument with the absolute values.

A: If I were following the same general route, I would prove by induction on $n$ that
$$0\le\sum_{k=1}^n(-1)^{k+1}a_{m+k}\le a_{m+1}$$
for each $m\in\Bbb Z^+$; since $a_1\ge a_2\ge\ldots\ge 0$, there is no need for absolute value signs.
This is clearly true for $n=1$. Suppose that it is true for some $n\ge 1$; we want to show that
$$0\le\sum_{k=1}^{n+1}(-1)^{k+1}a_{m+k}\le a_{m+1}\tag{0}$$
for any $m\in\Bbb Z^+$. By the induction hypothesis (applied to $m+1$ instead of $m$) we know that
$$0\le\sum_{k=1}^n(-1)^{k+1}a_{(m+1)+k}\le a_{m+2}\;,$$
which can be rewritten as
$$0\le\sum_{k=2}^{n+1}(-1)^ka_{m+k}\le a_{m+2}\;.\tag{1}$$
Now
$$\begin{align*}
\sum_{k=1}^{n+1}(-1)^{k+1}a_{m+k}&=a_{m+1}+\sum_{k=2}^{n+1}(-1)^{k+1}a_{m+k}\\
&=a_{m+1}-\sum_{k=2}^{n+1}(-1)^ka_{m+k}\;,
\end{align*}$$
and it follows from $(1)$ that
$$a_{m+1}-a_{m+2}\le a_{m+1}-\sum_{k=2}^{n+1}(-1)^ka_{m+k}\le a_{m+1}\;.$$
And since $a_{m+1}\ge a_{m+2}$, $(0)$ follows immediately.


Q: Cómo es la manera correcta de hacer for while en php? Estoy necesitando realizar un do while o for o una manera correcta para poder repetir contenido
<?php
mysqli_select_db($conexion, $database_conexion);
$query_DatosCategorias = sprintf("SELECT * FROM categorias WHERE estado>0");
$DatosCategorias= mysqli_query($conexion, $query_DatosCategorias) or die(mysqli_error());
$row_DatosCategorias = mysqli_fetch_assoc($DatosCategorias);
$totalRows_DatosCategorias = mysqli_num_rows($DatosCategorias);

$total = $totalRows_DatosCategorias;
$limite = 8;
$listas = null;
$inicio = 1;
$k = null;
$listas = $total/$limite;
$cantidadListas = ceil($listas);
?>
<footer class="site-footer footer-light"
  style="background-image: url(<?php echo $urlWeb;?>/img/footer-bg-alt.png);">

Categorias

<?php
for ($i=$inicio; $i <=$cantidadListas; $i++) {
  echo "
";
  for ($j=1; $j <=$limite && $k<$total; $j++) {
    $k = $k+1;
    //acá necesito hacer un bucle for o while pero no me da resultado
    echo "
";
    echo '
'.$row_DatosCategorias['id'].'
';
    echo "
";
  }
  echo "
";
}

A: En lugar del for, pon un while para recorrer el resultado. He sacado la etiqueta ul fuera porque he supuesto que los resultados son los items de la lista.
Modificado para que salgan varias listas.
$i=0;
while ($row_DatosCategorias = mysql_fetch_assoc($DatosCategorias)) {
  $i++;
  if ($i==1) {echo "
";} // primera lista
  echo '
'.$row_DatosCategorias['id'].'
';
  if ($i % $limite == 0 || $i == $total) {
    echo "
"; // cerrar lista
    // Si quedan más elementos abrir otra
    if ($i < $total) {echo "
";}
  }
}


Q: Only update messages when there is a connection I built a site display messages, so the msgloader.js updates the messages every 10 seconds from admin. The client displays this site on a screen constantly.
The problem is that sometimes the internet connection is not so good, the display got disconnected and the msgloader.js still updates the messages, in the end the screen freeze (the reason I know is that there is a clock on the site as well, the clock gets time from the local machine, it simply freeze at a time until we refresh the page which is an issue).
I suspect this freeze issue is because there is too much script running and the machine ram has been taken up.
BACK TO THE QUESTION, Is there any way we can set the below code into update messages every 10 seconds when there is an internet connection, otherwise update messages every hour/two when there is no internet connection.
Any help is appreciated. Thanks.
--------------------------------------------------------------------
Update the data for the static sections
--------------------------------------------------------------------
*/
function updateSection(sect)
{
    setTimeout('updateSection(' + sect + ')', 10000);

    //alert('updateSection: ' + sect);
    var ajax = new sack();
    ajax.requestFile = 'ajax/getMessages.php?section='+sect;
    ajax.method = 'post';
    /*ajax.onError = whenError;*/
    ajax.onCompletion = whenComplete;
    ajax.runAJAX();

/*  function whenError()
    {
        alert('Could not return getMessages values.
Please notify the system administrator.');
    }*/

    function whenComplete()
    {
        var messages = ajax.response;

        var messages1 = messages.split('---');

        var num_messages    = messages1[0];
        //alert('Num Lines: ' + num_messages );
        var messages_list   = messages1[1];
        //alert('MESSAGES: '+messages);
        var msg_data_array  = messages_list.split('::');

        var i=0;
        switch(sect)
        {
            case 1:

                for(i=0;i<=num_messages;i++)
                {

                    var j = i + 1;
                    icon_to_use = 'icon'+j;

                    // Set icon class
                    var icon = document.getElementById('icon_' + sect + '_' + j);
                    icon_to_use.className = 'icon_pointer';

                    // Set message text
                    // -------------------------------------------
                    var msgtext_array = msg_data_array[i].split('##');

                    // Here's the title
                    // -------------------------------------------
                    var msgtext_1a = msgtext_array[1];

                    // Here's the text
                    // -------------------------------------------
                    var msgtext_1 = msgtext_array[2];

                    // Set the title space
                    // -------------------------------------------
                    var msg_1a = document.getElementById('msg_text_' + sect + '_' + j + 'a');

                    // Set the text space
                    // -------------------------------------------
                    var msg_1 = document.getElementById('msg_text_' + sect + '_' + j);

                    // Write in the title
                    // -------------------------------------------
                    msg_1a.innerHTML = msgtext_1a;
                     msg_1.innerHTML = " " + msgtext_1;
                    // Write in the text
                    // -------------------------------------------
                    msg_1.innerHTML = (msgtext_1) ? separator + msgtext_1 : msgtext_1;

                    //msg_1a.style.borderBottom="2px solid white";
                    msg_1a.style.borderBottom="2px solid white";
                    msg_1.style.borderBottom="2px solid white";

                }   break;
            default:
                break;
        }

        // DEBUG
        if(debug)
        {
            debugReport
            (
                'updateSection():ID: '+msg_id+
                '
'+'updateSection():TIMEOUT: '+timeout+
                '
'+'ROTATE: '+rotate
            );
        }
        else
        {
            debugReset();
        }
    }
}
/*

A: Try to use this,
var online = navigator.onLine;

and now you can do like this,
if(online){
    alert('Connection is good');
}
else{
    alert('There is no internet connection');
}

UPDATE:
Try to put the alert here,
if(online){
    setTimeout('updateSection(' + sect + ')', 10000);

    //alert('updateSection: ' + sect);
    var ajax = new sack();
    ajax.requestFile = 'ajax/getMessages.php?section=1';
    ajax.method = 'post';
    /*ajax.onError = whenError;*/
    ajax.onCompletion = whenComplete;
    ajax.runAJAX();
}
else{
    alert('There is no internet connection');
}

A: If I'm understanding you correctly you could do something like this:
every time onerror event happens on an ajax request increment a counter. After a set limit of fails in a row / fails in an amount of time you change the length of the time-out.
var timeoutLength = 10000
setTimeout('updateSection(' + sect + ')', timeoutLength);

changing the timeoutLength once the ajax requests are failing, IE there is not internet connection.
EDIT
var errorCount = 0;

ajax.onError = whenError;

function whenError() {
  errorCount++
  if(errorCount < 5) {
    timeoutLength = 3600000
  }
}

function whenComplete() {
   errorCount = 0
   ...
}

This requires 5 errors in a row to assume that the internet is down. You should probably play around with that. But this should show you the general idea.


Q: "General::sysffmpeg: Using a limited version of FFmpeg" won't go away on Linux (MMA 12.3) This issue appeared when I tried to render animation videos
using Export on Fedora.
$VideoEncoders["MP4"][[All, 1]]
General::sysffmpeg: Using a limited version of FFmpeg. Install FFmpeg to get more complete codec support.

The Wolfram community forum has a post detailing the
installation of ffmpeg on Linux, but it was useless.
An internal variable can be used to check whether system ffmpeg is used:
FFmpegTools`$SystemFFmpegQ
False

ffmpeg 4.4.2 is installed and working and should be compatible  (4.4.2 >= 4.0.0), but Mathematica refuses to use it without
providing any meaningful error message to indicate what
the problem is, and
Internal`$LastInternalFailure

shows nothing either.

A: By hacking SystemFiles/Links/FFmpegTools/Kernel/Common.m I managed to get a meaningful error message.
The problem is that Mathematica bundles shared libraries which are incompatible with system shared libraries. In this case, ffmpeg depends on kerberos, which depends on openssl, which Mathematica bundles.
The solution is to force Mathematica to use the system library.
Removing or renaming the offending shared library is all it takes.
sudo mv Mathematica/12.3/SystemFiles/Libraries/Linux-x86-64/libcrypto.so.1.1 Mathematica/12.3/SystemFiles/Libraries/Linux-x86-64/libcrypto.so.1.1.backup

Restart the kernel and you should be ok.
FFmpegTools`$SystemFFmpegQ (* True *)

$VideoEncoders["MP4"][[All, 1]] (* Check that more codecs are available *)


Q: I am trying to scrape a website using python request that doesn't change its link when click on load more i tried but looks its json I'm trying to scrape all the add links from a site and have I been successful in doing so. please check the link
https://www.olx.in/hyderabad_g4058526/q-note-9-pro?isSearchCall=true
The site page has a load more button for loading more adds.
[1]: https://ibb.co/DGNsG6j"
The problem is that clicking on load more doesn't change the URL of the page, therefore I'm being able to scrape only the initial links displayed by default Please help me with this.

A: Have you considered using Selenium? It is able to click through websites automatically by locating HTML tags and clicking links. https://selenium-python.readthedocs.io/index.html

A: Look into their private API; Inspect Element, and Network tab. Notice when you click the 'Load More' button your browser makes a request to the following URL:
https://www.olx.in/api/relevance/v2/search?facet_limit=100&lang=en&latitude=17.46497&location=4058526&location_facet_limit=20&longitude=78.43517&page=1&platform=web-desktop&query=note%209%20pro&spellcheck=true

You can modify the query via the URL params. Then, using requests, you can query this information and organize the JSON return:
{
  "version": "100.0",
  "data": [
    {
      "id": "1636448838",
      "score": 1679.043212890625,
      "spell": {
        "id": 54,
        "key": "GEAS",
        "version": "1",
        "main": true,
        "facet_disabled": false,
        "default_sorting": "DEFAULT"
      },
      "status": {
        "status": "active",
        "allow_edit": true,
        "ban_reason_id": null,
        "display": "active",
        "translated_display": "Active",
        "link": null,
        "flags": {
          "new": false,
          "hot": false
        },
        "message": null
      },
      "category_id": "1453",
      "favorites": {
        "count": 0,
        "count_label_next": "1"
      },
      "images": [
        {
          "id": "1003855963",
          "external_id": "tj7vu6vj98qp-IN",
          "width": 524,
          "height": 1080,
          "url": "https://apollo-singapore.akamaized.net:443/v1/files/tj7vu6vj98qp-IN/image",
          "full": {
            "width": 1080,
            "height": 2225,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/tj7vu6vj98qp-IN/image;s=1080x2225"
          },
          "big": {
            "width": 505,
            "height": 1040,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/tj7vu6vj98qp-IN/image;s=505x1040"
          },
          "medium": {
            "width": 120,
            "height": 247,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/tj7vu6vj98qp-IN/image;s=120x247"
          },
          "small": {
            "width": 120,
            "height": 247,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/tj7vu6vj98qp-IN/image;s=120x247"
          }
        },
        {
          "id": "1003855964",
          "external_id": "s6z25cvcbz052-IN",
          "width": 524,
          "height": 1080,
          "url": "https://apollo-singapore.akamaized.net:443/v1/files/s6z25cvcbz052-IN/image",
          "full": {
            "width": 1080,
            "height": 2225,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/s6z25cvcbz052-IN/image;s=1080x2225"
          },
          "big": {
            "width": 505,
            "height": 1040,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/s6z25cvcbz052-IN/image;s=505x1040"
          },
          "medium": {
            "width": 120,
            "height": 247,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/s6z25cvcbz052-IN/image;s=120x247"
          },
          "small": {
            "width": 120,
            "height": 247,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/s6z25cvcbz052-IN/image;s=120x247"
          }
        },
        {
          "id": "1003855965",
          "external_id": "mq6ouyke1k3h3-IN",
          "width": 524,
          "height": 1080,
          "url": "https://apollo-singapore.akamaized.net:443/v1/files/mq6ouyke1k3h3-IN/image",
          "full": {
            "width": 1080,
            "height": 2225,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/mq6ouyke1k3h3-IN/image;s=1080x2225"
          },
          "big": {
            "width": 505,
            "height": 1040,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/mq6ouyke1k3h3-IN/image;s=505x1040"
          },
          "medium": {
            "width": 120,
            "height": 247,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/mq6ouyke1k3h3-IN/image;s=120x247"
          },
          "small": {
            "width": 120,
            "height": 247,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/mq6ouyke1k3h3-IN/image;s=120x247"
          }
        },
        {
          "id": "1003855966",
          "external_id": "7vfcbj5fb8fh2-IN",
          "width": 524,
          "height": 1080,
          "url": "https://apollo-singapore.akamaized.net:443/v1/files/7vfcbj5fb8fh2-IN/image",
          "full": {
            "width": 1080,
            "height": 2225,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/7vfcbj5fb8fh2-IN/image;s=1080x2225"
          },
          "big": {
            "width": 505,
            "height": 1040,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/7vfcbj5fb8fh2-IN/image;s=505x1040"
          },
          "medium": {
            "width": 120,
            "height": 247,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/7vfcbj5fb8fh2-IN/image;s=120x247"
          },
          "small": {
            "width": 120,
            "height": 247,
            "url": "https://apollo-singapore.akamaized.net:443/v1/files/7vfcbj5fb8fh2-IN/image;s=120x247"
          }
        }
      ],
      "certified_car": false,
      "is_kyc_verified_user": false,
      "has_phone_param": false,
      "locations_resolved": {
        "COUNTRY_id": "1000001",
        "COUNTRY_name": "India",
        "ADMIN_LEVEL_1_id": "2007599",
        "ADMIN_LEVEL_1_name": "Telangana",
        "ADMIN_LEVEL_3_id": "4058526",
        "ADMIN_LEVEL_3_name": "Hyderabad",
        "SUBLOCALITY_LEVEL_1_id": "5348837",
        "SUBLOCALITY_LEVEL_1_name": "Murad Nagar"
      },
      "description": "Full condition with box + bill charger",
      "created_at": "2021-05-18T10:53:54+05:30",
      "inspection_info": null,
      "package_id": null,
      "title": "Note 9 pro",
      "main_info": null,
      "user_type": "Regular",
      "display_date": "2021-05-18T05:23:54+0000",
      "user_id": "508731436",
      "price": {
        "value": {
          "raw": 14500,
          "currency": {
            "iso_4217": "INR",
            "pre": "₹"
          },
          "display": "₹ 14,500"
        },
        "key_name": "Price",
        "key": "price"
      },
      "created_at_first": "2021-05-18T10:53:54+05:30",
      "locations": [
        {
          "lat": 17.39,
          "lon": 78.448,
          "region_id": "2007599",
          "district_id": "5348837",
          "city_id": "4058526"
        }
      ],
      "parameters": [
        {
          "type": "single",
          "key": "make",
          "value": "mi-phone",
          "key_name": "Brand",
          "formatted_value": "Mi",
          "value_name": "Mi"
        }
      ],
      "monetizationInfo": null
    }
  ],
  "metadata": {
    "sections": [
      {
        "id": 10,
        "offset": 0,
        "criterion": "less_than",
        "distance": 10,
        "relaxation_type": "bucket"
      },
      {
        "id": 20,
        "offset": 39,
        "criterion": "less_than",
        "distance": 20,
        "relaxation_type": "bucket"
      }
    ],
    "feed_version": "100.0",
    "total_ads": 190,
    "total_suggested_ads": 0,
    "ads_on_page": 20,
    "total_pages": 4,
    "suggested_sections": [],
    "original_term": "note 9 pro",
    "modified_term": "note 9 pro",
    "original_label": "Showing results for ${original_term}",
    "show_hint": false,
    "show_original_items": true,
    "show_suggested_items": false,
    "filters": [
      {
        "id": "price",
        "values": [],
        "display_order": 200,
        "description": "Budget",
        "range": [
          {
            "id": "min",
            "description": "min",
            "min_value": 0,
            "max_value": 10000000000
          },
          {
            "id": "max",
            "description": "max",
            "min_value": 0,
            "max_value": 10000000000
          }
        ],
        "type": "range-input",
        "render_as": "range-input"
      }
    ],
    "applied_sorting": {
      "key": "desc-creation"
    },
    "next_page_url": "http://api.olx.in/relevance/v2/search?facet_limit=100&clientId=pwa&location_facet_limit=20&spellcheck=true&latitude=17.46497&query=note%209%20pro&location=4058526&page=2&lang=en&clientVersion=7.10.1&platform=web-desktop&longitude=78.43517",
    "applied_filters": [],
    "search_query": "note 9 pro",
    "modified_filters": {}
  },
  "empty": false,
  "not_empty": true,
  "suggested_data": []
}

I deleted all item entries except one above.


Q: Trouble configuring Presto's memory allocation on AWS EMR I am really hoping to use Presto in an ETL pipeline on AWS EMR, but I am having trouble configuring it to fully utilize the cluster's resources.  This cluster would exist solely for this one query, and nothing more, then die.  Thus, I would like to claim the maximum available memory for each node and the one query by increasing query.max-memory-per-node and query.max-memory.  I can do this when I'm configuring the cluster by adding these settings in the "Edit software settings" box of the cluster creation view in the AWS console.  But the Presto server doesn't start, reporting in the server.log file an IllegalArgumentException, saying that max-memory-per-node exceeds the useable heap space (which, by default, is far too small for my instance type and use case).
I have tried to use the session setting set session resource_overcommit=true, but that only seems to override query.max-memory, not query.max-memory-per-node, because in the Presto UI, I see that very little of the available memory on each node is being used for the query.
Through Google, I've been led to believe that I need to also increase the JVM heap size by changing the -Xmx and -Xms properties in /etc/presto/conf/jvm.config, but it says here (http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto.html) that it is not possible to alter the JVM settings in the cluster creation phase.
To change these properties after the EMR cluster is active and the Presto server has been started, do I really have to manually ssh into each node and alter jvm.config and config.properties, and restart the Presto server?  While I realize it'd be possible to manually install Presto with a custom configuration on an EMR cluster through a bootstrap script or something, this would really be a deal-breaker.
Is there something I'm missing here?  Is there not an easier way to make Presto allocate all of a cluster to one query?

A: As advertised, increasing query.max-memory-per-node, and also by necessity the -Xmx property, indeed cannot be achieved on EMR until after Presto has already started with the default options.  To increase these, the jvm.config and config.properties found in /etc/presto/conf/ have to be changed, and the Presto server restarted on each node (core and coordinator).
One can do this with a bootstrap script using commands like
sudo sed -i "s/query.max-memory-per-node=.*GB/query.max-memory-per-node=20GB/g" /etc/presto/conf/config.properties
sudo restart presto-server

and similarly for /etc/presto/jvm.conf.  The only caveats are that one needs to include the logic in the bootstrap action to execute only after Presto has been installed, and that the server on the coordinating node needs to be restarted last (and possibly with different settings if the master node's instance type is different than the core nodes).
You might also need to change resources.reserved-system-memory from the default by specifying a value for it in config.properties.  By default, this value is .4*(Xmx value), which is how much memory is claimed by Presto for the system pool.  In my case, I was able to safely decrease this value and give more memory to each node for executing the query.

A: As a matter of fact, there are configuration classifications available for Presto in EMR. However, please note that these may vary depending on the EMR release version. For a complete list of the available configuration classifications per release version, please visit 1 (make sure to switch between the different tabs according to your desired release version). Specifically regarding to jvm.config properties, you will see in 2 that these are not currently configurable via configuration classifications. That being said, you can always edit the jvm.config file manually per your needs.
Amazon EMR 5.x Release Versions
1
Considerations with Presto on Amazon EMR  - Some Presto Deployment Properties not Configurable:
2


Q: Recursive SELECT in Oracle I have two tables:
Table One
NumLig - CodPro - CodDer
901 - BSFX30 - 0140I18
898 - RSFX30 - 18
6 - MFLX - U

Table Two
NumLig - CodCmp - DerCmp - QtdUti - PerCmp
6 - MDFIB0008 - null - 1 - 2
6 - MDQDN0009 - null - 0,24 - 1
898 - MFLX - U - 0,942 - 2
898 - MDCAM0002 - 0,05 - 0
901 - RSFX - 18 - 1 - 2,5
901 - EDEAD0005 - 0,245 - 0

What I need is that I search for BSFX30 - 0140I18 the output to be:
MDFIB0008 - null - 1 - 2
MDQDN0009 - null - 0,24 - 1
MFLX - U - 0,942 - 2
MDCAM0002 - 0,05 - 0
RSFX - 18 - 1 - 2,5
EDEAD0005 - 0,245 - 0

If I do this:
SELECT E622SIM.CodPro, E622SIM.CodDer, E622DER.TipCpc, E622SIM.NumLig, E622DER.CodCmp, E622DER.DerCmp, E622DER.QtdUti, E622DER.PerCmp
FROM E622SIM
JOIN E622DER
ON E622SIM.NumLig = E622DER.NumLig

The output to be all the data, but I need filter by CodPro and CodDer. If I do this:
SELECT E622SIM.CodPro, E622SIM.CodDer, E622DER.TipCpc, E622SIM.NumLig, E622DER.CodCmp, E622DER.DerCmp, E622DER.QtdUti, E622DER.PerCmp
FROM E622SIM
JOIN E622DER
ON E622SIM.NumLig = E622DER.NumLig

the output to be only:
RSFX - 18 - 1 - 2,5
EDEAD0005 - 0,245 - 0

but I need:
MDFIB0008 - null - 1 - 2
MDQDN0009 - null - 0,24 - 1
MFLX - U - 0,942 - 2
MDCAM0002 - 0,05 - 0
RSFX - 18 - 1 - 2,5
EDEAD0005 - 0,245 - 0

This because RSFX is children of BSFX30, and MFLX is children of RSFX
Thanks!

A: I think you need see documentation about Hierarchical Queries
to connect your data use connect by, to get root element - connect_by_root, to filter the elements by root you need use where connect_by_root(E622SIM.CodPro)  = 'BSFX30'
I tried to compile your description into scripts
create table E622SIM(NumLig int, CodPro varchar2(100), CodDer varchar2(100));

insert into E622SIM values (901, 'BSFX30', '0140I18');
insert into E622SIM values (898, 'RSFX30', '18');
insert into E622SIM values (6, 'MFLX', 'U');

create table E622DER (NumLig int, CodCmp varchar2(100), DerCmp varchar2(100), QtdUti varchar2(100), PerCmp varchar2(100));
insert into E622DER values(901, 'RSFX30',18,1,'2,5');
insert into E622DER values(901, 'EDEAD0005',0,245,0);
insert into E622DER values(898, 'MFLX','U','0,942',2);
insert into E622DER values(898, 'MDCAM0002',null,'0,05',0);
insert into E622DER values(6, 'MDFIB0008', null, 1, 2);
insert into E622DER values(6, 'MDQDN0009',null,'0,24',1);

and this is the query that looks as what you need
select E622DER.CodCmp
      ,E622DER.DerCmp
      ,E622DER.QtdUti
      ,E622DER.PerCmp
from E622DER
  left outer join E622SIM on E622SIM.NumLig = E622DER.NumLig
  where connect_by_root(E622SIM.CodPro)  in ('BSFX30')
   connect by prior E622DER.CodCmp = E622SIM.CodPro
order by level desc, E622DER.CodCmp asc

    CODCMP  DERCMP  QTDUTI  PERCMP
1   MDFIB0008       1   2
2   MDQDN0009       0,24    1
3   MDCAM0002       0,05    0
4   MFLX    U   0,942   2
5   EDEAD0005   0   245 0
6   RSFX30  18  1   2,5


Q: Is CDN and Cloud are same? Is the CDN and Cloud computing are same? If both are different, how they differ? I have googled it, when someone asked me this question. But i could not able to get any clear idea on this.  So can anyone explain about this, which would be greatly useful to lot of people.
And also please explain where the CDN will be used and where the cloud will be used?

A: The big difference is that cloud computing is a big group of servers in 1 data center building which is usually at one location. On the other hand CDN is also group of servers but distributed around the country so it allows web visitors a better and faster access to the website. For example if you're in A location trying to access a server in B it can be faster to be hitting a server locally in A for the files. The CDN is usually able to support much larger traffic volumes since the speed is calculated based on location the traffic comes from.
CDN work on the principle of delivering content form the nearest located server as per user location.
CDN is short is a way to boost and speed up your website turn around time.

A: CDN is simply a network of servers that replicate your binary files so that they are served from geographically close locations. CDN has been around for a lot longer than cloud computing as you know it today.
Not every cloud provider is a CDN, and not every CDN is a cloud computing provider.
Cloud computing is simply - dividing up a large computing resource (usually processing power) into little chunks which you can use remotely.
CDN is simply - a bunch of "disks" that are spread across the world in different datacenters. You upload your file to one of these disks - and then tell it where your customers are coming from. It will then copy the same file to other disks that are nearer to your customers; giving your visitors a faster experience. This collection of disks is called the content delivery network.
One of the biggest names in CDN is Akamai.

A: Short answer: They are different. Detailed one follows:

*

*CDN short for Content delivery network is more like edge computing. It follows end-to-end principle of networking meaning, as much work can be de-centralized and distributed to nodes near user, do it. Reduce single point of failure. You can read a small article written by me at : http://www.sitepoint.com/content-delivery-networks-cdn-get-to-the-edge/

*Cloud Computing is much more than delivering content near edge. It's elastic computing, storage and network on demand in very broad sense. For computing you need: storage & processing power and that's what is provided by Cloud Computing


Q: jQuery countTo with SharePoint So I have a custom html/aspx in my sharepoint page with a counter at the bottom part of my page.
If the 'data-to' attribute of my div is hard coded

and when previewing the page, it works as expected.
but when I have a JS that will retrieve the ItemCount of a list using REST API
and set the 'data-to' based on the ItemCount I got from the REST API,
it doesn't work even if I inspect the element and 'data-to' has an updated value based on the retrieved ItemCount
function GetAllCount() { //stats
var list        = _spPageContextInfo.webAbsoluteUrl + "/_api/web/lists/GetByTitle('Test')/ItemCount";
var itemCount   = 0;

$.ajax({
    url: list,
    method: "GET",
    headers: {
        "accept":"application/json;odata=verbose"
    },
    success: function(data) {
        console.log(data);
        itemCount = data.d.ItemCount;
        $('#test1').attr('data-to', 1000);
    },
    error: {

    }
})

}

A: In your example, the jQuery CountTo is only be processed at load time, and before your AJAX completes. Changing the attribute after the timer has run will not restart it.
See the "JavaScript Options" section of https://github.com/mhuggins/jquery-countTo#javascript-options and run code like this in your AJAX success function:
$('.timer').countTo({from: 0, to: 500});


Q: Minify JS Files with Webpack I'm trying to use the webpack to minify all my js, .css, images and fonts files. As I am studying the Webpack, I am trying first with the JS files.
In my js folder I have the files jquery-2.2.3.min.js, bootstrap.min.js, easing.js, move-top.js and SmoothScroll.min.js.
when i try to run npm start it generates the error Error: Conflict: Multiple chunks iss assets to the same filename principal.js (chunks jquery and bootstrap).
Below is my webpack.config.js:

const webpack = require('webpack');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');

module.exports = {
    mode: 'development',
    entry: {
        jquery: './js/jquery-2.2.3.min.js',
        bootstrap: './js/bootstrap.min.js',
        easing: './js/easing.js',
        move_top: './js/move-top.js',
        smooth_scroll: './js/SmoothScroll.min.js',
    },
    output: {
        filename: 'default.js',
        path: __dirname + '/public'
    },
    plugins: [
        new MiniCssExtractPlugin({
            filename: "style.css"
        })
    ],
    module: {
        rules:[
                {
                    test: /\.css$/,
                    use: [
                        MiniCssExtractPlugin.loader,
                        'css-loader'
                    ]
                },
                {
                    test: /\.(png|svg|jpg|jpeg|gif)$/,
                    use: [
                        'file-loader'
                    ]
                }
            ]
        }
}


Q: What's the easiest way to gain the Master of Patience achievement? One of the Federation Cruiser's ship-specific achievements is Master of Patience:

Use only the Artillery Beam to destroy an enemy ship while taking no hull damage.

What is the quickest/easiest/safest way to gain this achievement?

A: I've found that this can easily be picked up in the first couple of sectors.
All you really need is 50 scrap to upgrade your shield subsystem two four bars (two shield "bubbles" when fully powered).  In general encounters, ignore this upgrade and leave it unpowered until you find an enemy ship that cannot get through two shields.  This generally involves them having one weapon which takes a single shield off (basic laser, heavy laser, anti-ship drone) and any number of beam weapons (which are useless against shields*).
Once you've found such an enemy, it's simply a case of moving power from your Burst Laser to your second shield layer - and waiting for your Artillery Beam to destroy them.
This is probably easier to pull off early, because it's less likely that you can reliably soak all enemy damage in later sectors, even with fully-upgraded shields.

*I don't know whether enemies can have beam weapons that do more than one damage per section, and hence could penetrate a single shield, but I've not seen them.  And it's safe to assume they wouldn't appear early in any case.

A: I strongly suggest doing it in early system (1-3) at encounter with asteroids. If you have a bit of luck most of the work will be done for you.
Don't worry - damage from asteroids did most of the damage for me and I earned the achievement.


Q: Define a function to update a class integer property I'm having issues defining a function that controls the correct "speed" (accelerate/slow down).
I currently have my code like this:
class Train():

    serial_nummer = 2021001

    def __init__(self, type, buildyear):
        self.type = type
        self.buildyear = buildyear
        self.serial_nummer = Train.serial_nummer
        self.speed = ()
        Train.serial_nummer += 1
        return

    def snelheid(self, speed):
        pass

    def __str__(self):
        return (f"Train type: {self.type} \nBuildyear: {self.buildyear} \nSerial number: {self.serial_nummer} \nCurrent speed: {self.speed}\n\n")

train1 = Train('Alfatrain', '2001')
train1.speed = 100
print (train1)

How can I create a function that controls the correct "speed"?
Now I'm just modifying the speed with train.speed = 100.

A: You don't really need a function when you can use train.speed += amount. You will want to initialize the speed as 0, not an empty tuple, though
Without more clarity, I'm guessing instructions are looking for
def accelerate(self, amount):
    self.speed += amount

def decelerate(self, amount):
    self.accelerate(-1*amount)


Q: Convert the string to integer or number How can I convert the string in to the number/integer inside the transfor dataweave. I tried the below
%dw 2.0
import * from dw::util::Coercions
output application/json
---
{
   "quoteId" : vars.setQuoteOppRecIds.Id,
   "productCode" : payload.ServiceTypeCode,
   "axSequenceNumber" : vars.counter as :number,
   "phaseLevel" : payload.PhaseLevel as :number,
   "activeInSOW" : if(payload.PhaseLevelActivateInSOW == "Yes") (toBoolean("true")) else (toBoolean("false")),
   "phaseLevelProject" : payload.PhaseLevelProject
}

But I get the error like Invalid input ':', expected } or ',' for the object expression. (line 8, column 41): I tried to string to boolean using the toBoolean function and itseems to be working fine.  Can anyone please tell me what is that I am missing with the string to integer/number conversion

A: The syntax for conversion in DW 2 is different. The Code you used is from dw 1. Adding references to type conversions in dw 2 below and fixed your DW script as well.

https://docs.mulesoft.com/dataweave/2.1/dataweave-types

 %dw 2.0
import * from dw::util::Coercions
output application/json
---
{
   "quoteId" : vars.setQuoteOppRecIds.Id,
   "productCode" : payload.ServiceTypeCode,
   "axSequenceNumber" : vars.counter as Number,
   "phaseLevel" : payload.PhaseLevel as Number,
   "activeInSOW" : if(payload.PhaseLevelActivateInSOW == "Yes") (toBoolean("true")) else (toBoolean("false")),
   "phaseLevelProject" : payload.PhaseLevelProject
}


Q: Requiring array of modules in Webpack I am currently trying to load a variable array of modules with Webpack.
I've found this link: webpack can not require variable ,the request of a dependency is an expression but it doesn't seem to work for me.
In my project I've got a module inside my node_modules folder. The entry point of the module is called index.js.
Here's the basic structure:
| app
+---app.js
| js
+---gen.js
| node_modules
  +---sample_module_1
  |   +---index.js
  +-sample-module_2
      +---index.js
| webpack.config.js

In future I'd like to add new modules. Therefore I tried following approach:
//app.js

var modules = [
  "sample_module_1",
  "sample_module_2"
]

for(var i = 0; i < modules.length; i++) {
  require(modules[i] + "/index.js");
}

But Webpack doesn't seem to find the module. I've tried adding a resolveLoader to the webpack.config.js file:
//webpack.config.js

var path = require('path');

module.exports = {
  entry: './app/app.js',
  output: {
    filename: 'gen.js',
    path: path.resolve(__dirname, 'js')
  },
  resolveLoader: {
    modules: ["node_modules"],
    extensions: ['*', '.js']
  }
};

Still, Webpack is not able to find the module.
I've also tried the suggestions on https://webpack.github.io/docs/context.html but still no results.
Any suggestions?


Q: Why are message view counts laggy on most web sites? Notice how view counts of youtube videos are always laggy? For example, a video has like 1000 comments and still has 500 hits, and will have 10000 hits hours after.
Youtube isn't alone in this. Most message boards are implemented that way and view counts are updated like every 10 minutes or so.
Does anyone know the reason behind this?
Thanks.

A: Most likely the value has been cached somewhere along the way so you are seeing stale data. Because it is not critical for this data to be accurate the developers have decided to favour performance over getting up to date data. You really wouldn't want to go to the database and do a row count for every hit on the site just to update this figure so they don't, they just cache it for a while.

A: In order for large sites to scale, they have to perform caching at several stages.  That can be page caching, sub-page caching, and/or record caching.  You might have a combination of all of them in effect.  For example, if the youtube page is cached until a new comment is added, you'll see some lag until someone posts a comment.
There are several ways of measuring page views:

*

*Store it in the database as a record: easy to insert, however it is a major maintenance overhead for records that are only providing a count.

*Store it in the database as a record and roll up the counts periodically: easy to insert, batch processing to gather the stats you want, and cleans up after itself.

*Update a count column in the database: expensive to update (assuming row locking), no maintenance overhead, negative performance when dealing with multiple people requesting the same page at the same time.

*Process the access log file when it rolls over: no extra data in the database, all processing is done in batches off-line, and the summary stats you want are updated when it is time.

Out of the items above, all except one option suggests that the updates will be done in batches.  The number of views is not really a time critical attribute, so this is OK.  However, keeping people waiting to view a video on YouTube because the backend database can't keep up is a time critical action.  That means that updating a column in the database isn't going to work for a site as large as YouTube.  I personally wouldn't be surprised if they opted for the final option.  The web servers will be recording a whole host of information for every visit including what IP you are using, how you were referred to the page, etc.  It only makes sense to process those in batches and summarize the results as necessary.

A: Recording views is very simple, simply add a row to a table that represent the "view" action. This is fast because no locking is required in the database, you're just adding a row onto the end of a heap.
Aggregating that into the total number of views requires something like doing SELECT COUNT(*) FROM ... which means you have to lock the table while the calculation is progressing. Alternatively, UPDATE ... SET num_views = num_views + 1 also requires that you lock that particular row every time someone views it.
So from a scalability point of view, it's much more efficient to add a row each time someone views the video and then do the SELECT COUNT(*) FROM ... every ten minutes or so.
Note I don't actually know the architecture of YouTube, or whether they even use a relational database to store their data, but whatever they do use, the principle is likely the same: inserting data is cheap, aggregating values is (relatively) expensive.

A: This could be due to a number of reasons.  It all boils down to the algorithms used by each respective website.  Unless someone here is actually a YouTube developer I doubt your going to get an exact answer here.


Q: Entity Framework Core: What is the fastest way to check if a generic entity is already in the dbset and then add or update it? I load data from a CSV file, I create entities on the fly then I add them to a List<T>.
The following code takes a List<T> and adds the entities to the right DbSet.
public static void AddEntities<T>(List<T> entities, DbContext db) where T :class
{
    using (db)
    {
        var set = db.Set<T>();

        foreach(T e in entities)
        {
            set.Add(e);
        }
        db.SaveChanges();
     }
}

I'd like to change it to add the entity only if it doesn't exist, otherwise it has to update it.
What is the best way to accomplish this with Entity Framework Core?
I believe I should use System.Reflection to:

*

*get at least the ID but it would be better to get all the keys for the dbset

*loop through the keys to find if an entity is already in the DbSet

*if found then update it with the values from the new entity otherwise add it to the set

Something like this:
public static void AddEntities<T>(List<T> entities, DbContext db) where T :class
{
    using (db)
    {
        var set = db.Set<T>();

        foreach(T e in entities)
        {
            var idProperty = e.GetType().GetProperty("ID").GetValue(e,null);
            var obj = set.Find(idProperty);

            if (obj==null)
            {
                set.Add(e);
            }
            else
            {
                var properties = (typeof(T)).GetProperties();

                foreach (var p in properties)
                {
                    var value = e.GetType().GetProperty(p.Name).GetValue(e,null);
                    obj.GetType().GetProperty(p.Name).SetValue(obj,value);
                 }
            }

        }
        db.SaveChanges();
    }
}

The code runs from 3 to 4 times slower than the simple add.
Is there a faster way? All the code examples I am finding are all for EF6, based on ObjectContext and IObjectContextAdapter and it seems this kind of code do not longer work with EF Core.

A: Instead of reflection, you could use the EF Core public (and some internal) metadata services to get the key values needed for Find method. For setting the modified values you could use EntityEntry.CurrentValues.SetValues method.
Something like this:
using Microsoft.EntityFrameworkCore.Metadata.Internal;

public static void AddEntities<T>(List<T> entities, DbContext db) where T : class
{
    using (db)
    {
        var set = db.Set<T>();

        var entityType = db.Model.FindEntityType(typeof(T));
        var primaryKey = entityType.FindPrimaryKey();
        var keyValues = new object[primaryKey.Properties.Count];

        foreach (T e in entities)
        {
            for (int i = 0; i < keyValues.Length; i++)
                keyValues[i] = primaryKey.Properties[i].GetGetter().GetClrValue(e);

            var obj = set.Find(keyValues);

            if (obj == null)
            {
                set.Add(e);
            }
            else
            {
                db.Entry(obj).CurrentValues.SetValues(e);
            }
        }
        db.SaveChanges();
    }
}


Q: Regex extracting image links I've got the following Regular Expression to extract links out of an HTML document, using java.util.regex
 ]+).*?.*?<\/a>)

and suspect it to match the last link in this markup.
Mail : s.weber@pgt-gmbh.com

Internet : http://www.pgt-gmbh.com

  - Anfrage.pdf

But it doesn't match the link but does something like a greedy search, starting with the mailto: and ending with the last link. The same expression works fine with the regex tester within http://regex101.com.
Any hints?

A: The problem would not occur if newlines are at the end of the text lines.
Now I have an explanation: The <a href="mailto is matched by the regular expression ]+). The following .*? will match any character sequence (without line breaks) until it finds <img.... And it does exactly this (in absence of line breaks).
Example (one with and one without newlines):
private static final Pattern P = Pattern.compile("]+).*?.*?<\\/a>)");
private static final String TEXT = "Mail              : s.weber@pgt-gmbh.com
"
    + "
"
    + "Internet        : http://www.pgt-gmbh.com
"
    + "
"
    + "
"
    + "  - Anfrage.pdf";
private static final String NEWLINE_TEXT = "Mail              : s.weber@pgt-gmbh.com
\n"
    + "
\n"
    + "Internet        : http://www.pgt-gmbh.com
\n"
    + "
\n"
    + "
\n"
    + "  - Anfrage.pdf";

public static void main(String[] args) {
    Matcher m = P.matcher(TEXT);
    if (m.find()) {
        System.out.println(m.group());
    }
    m = P.matcher(NEWLINE_TEXT);
    if (m.find()) {
        System.out.println(m.group());
    }
}

Output:
... without newlines
<a class="domino-attachment-link"... with newlines
A better pattern
]*?href=([^>]+)>.*?<\/a>)
The problem with HTML and regex is that the upper pattern matches only a specific situation, if some markup is between  and  then it would fail. Surely this could be fixed, but the expression gets more and more incomprehensible.
So: If you want to do this extraction issues for more than one link, you should switch to an HTML-Parser (although finding the best is a science of its own).


Q: Can't add component to JPanel in Swing I want to add at runtime a JLabel under the purple pane which contains already some components, say under the progress bar:

Here is the structure of the elements:

And this is my code which is issued when an event occurs (it's getting there i checked with debug) :
    jPanel1.add(new JLabel("Stack Overflow"));
    jPanel1.revalidate();
    jPanel3.revalidate();

I'm not seeing any changes whatsoever and have no clue where to go from here. When i put a
textarea in the purple pane and then call it's setText() method at the same place i try to add the JLabel component it works.

A: You need to learn more about layouts and how they work. I strongly suggest you read the entire layout manager tutorial, since understanding layouts are the solution here, and just using BorderLayout isn't the way to solve it. You'll likely want to nest layouts, perhaps using BorderLayout for the overall GUI, and having a central JPanel use BoxLayout to allow you to stack components on top of each other inside of it. Then perhaps add this JPanel to the main JPanel that uses BorderLayout in the BorderLayout.CENTER position.

A: Just a hunch, but maybe you need to call repaint() in addition to revalidate()
Java Swing revalidate() vs repaint()


Q: JavaFX close dialog on thread I am trying  to close JavaFX custom dialog after some job is done, but It's not closing. I have tried everything what was on web. But still no success.
I am using :
  import javafx.scene.control.Dialog;

Code:
 dialogProgressBar.progressIndicator().show();
 Service<Void> service = new Service<Void>() {
                @Override
                protected Task<Void> createTask() {
                    return new Task<Void>() {
                        @Override
                        protected Void call() throws Exception {
                            //Background work
                            mySqlDatabaseHandler.mySQLConnection();
                            mySqlDatabaseHandler.executeStatement();
                            closeConnection();
                            Platform.runLater(() -> {
                                //FX Stuff done here
                                dialogProgressBar.progressIndicator().close();
                            });

                            return null;
                        }
                    };
                }
            };
  service.start();


Q: replace XML tags value in shell script Hi I am a xml file like this.
  <parameter name="END_DATE">20181031</parameter>

I want to replace this tags value to some other value I tried like this.
dt=$(awk -F '[<>]' '/_DATE/{print $3}' test.xml)

I extracted the tags value.
I have another variable value like this.
newdt=20181108

Now I need to replace this value to the extracted value.
Any help would be appreciated.

A: Though Chepner is right that awk or sed are not exact tools for xml in case you are NOT having xmlstarlet in your system then try following.
echo $newdt
20181108
awk -v dat="$newdt" 'match($0,/>[0-9]+</){$0=substr($0,1,RSTART) dat substr($0,RSTART+RLENGTH-1)} 1' Input_file

A: If sed works for you -
sed -Ei 's/( name="END_DATE")>20181031</\1>20181108</' test.xml

And xml parser is probably a better idea, though.
If you need to embed the variable -
sed -Ei "s/( name=\"END_DATE\")>20181031</\1>$newdt</" test.xml


Q: Is google drive api for c# thread safe? I'm using google drive API in a multi-thread C# application. I would like to know if the Google dot net client library it's thread-safe or not.
Also I would like to know what's more right:
create a singelton service, or a new service every time.

A: If you are asking if the Google .net client Library is thread safe. I am pretty sure that it is.  It is noted in at least one place in the documentation that it is thread safe.
Google APIs Client Library for .NET

UserCredential is a thread-safe helper class for using an access token
  to access protected resources. An access token typically expires after
  1 hour, after which you will get an error if you try to use it.

A: It isn't.  I've tried writing some code assuming it was, and my code would sometimes randomly crash at the "request.Execute()"s...
I tried using a mutex and so only one thread was using a singleton service at a time: that reduced the crashes but it didn't eliminate them.
Leaving the mutex in, I changed it to a singleton UserCredential and use a new service for each thread and it hasn't crashed since.  (I don't know if I need the mutex anymore or not, but the google api calls aren't critical path for my application, I just needed to get the calls out of the main thread.)


Q: How to pass a data in WkWebView URL in swift In my project I have a dictionary. I need to pass it to a WKWebView as a post parameter. When i try to convert String to URL it returns nil value. any advice to me?
This is my Dictionary named CartDict.
{
  "address_id" : "4064",
  "customer_id" : "3239",
  "language_id" : "1",
  "products" : [
    {
      "option" : "",
      "product_id" : "1576",
      "quantity" : "2"
    },
    {
      "option" : "",
      "product_id" : "1573",
      "quantity" : "1"
    },
    {
      "option" : "",
      "product_id" : "1575",
      "quantity" : "1"
    }
  ],
  "set_currency" : "EUR"
}

This is my code:
let urlStr = "https://test.com/index.php?route=checkout/checkout_mobile&data=\(cartDict)"

let trimmedUrl = urlStr.trimmingCharacters(in: CharacterSet(charactersIn: "")).replacingOccurrences(of: " ", with: "%20")
let url = URL(string: trimmedUrl)
print(url)
self.webView.load(URLRequest(url: url!))

Thanks in advance.

A: Set httpBody to your data from dictionary
let trimmedUrl = urlStr.trimmingCharacters(in: CharacterSet(charactersIn: "")).replacingOccurrences(of: " ", with: "%20")
let url = URL(string: trimmedUrl)!
let dic:[String:Any] = ["address_id" : "4064", "customer_id" : "3239", "language_id" : "1", "products" : [ [ "option" : "", "product_id" : "1576", "quantity" : "2" ], [ "option" : "", "product_id" : "1573", "quantity" : "1" ], [ "option" : "", "product_id" : "1575", "quantity" : "1" ] ], "set_currency" : "EUR" ]
let data = try? JSONSerialization.data(withJSONObject: dic)
var req = URLRequest(url: url)
req.httpBody =   data
self.webView.load(req)

A: Firstly you need to make sure that the url string is valid, try printing it
print(urlStr)
The post parameters are not url params usually, but http body parameters, thus need to be added to the body of the request. You need to know what format the server is expecting. Modern apis use JSON so you'd need to format your dictionary to json (however the dictionary you posted is actually not a swift dictionary, so it looks ok.).
    let urlStr = "https://test.com/index.php?route=checkout/checkout_mobile"

    let trimmedUrl = urlStr.trimmingCharacters(in: CharacterSet(charactersIn: "")).replacingOccurrences(of: " ", with: "%20")
    let url = URL(string: trimmedUrl)
    let request = URLRequest(url: url)
request.httpBody = Data(cartDict.utf8)
    self.webView.load(request)

A: From the comments

when i convert this String (urlStr) to URLRequest it returns nil

If this is your question or it maybe useful for others in future.
extension URL {
   public var queryParameters: [String: Any]? {
      guard let components = URLComponents(url: self, resolvingAgainstBaseURL: true), let queryItems = components.queryItems else { return nil }
      return queryItems.reduce(into: [String: String]()) { (result, item) in
          result[item.name] = item.value
      }
   }
}

In your WKNavigationDelegate methods on redirection
func webView(_ webView: WKWebView, decidePolicyFor navigationAction: WKNavigationAction, decisionHandler: @escaping (WKNavigationActionPolicy) -> Void) {
  guard let url = navigationAction.request.url else {
      decisionHandler(.allow)
      return
  }
  if url.absoluteString.lowercased().range(of: "redirection/url/you/expected/to/intercept") != nil {
      var parameters: [String: Any]?
      if let requestBody = navigationAction.request.httpBody {
          //If your data is JSON
          let requestJSON = try? JSONSerialization.jsonObject(with: requestBody, options: [])
          if requestJSON = requestJSON as? [String: Any] {
              parameters = requestJSON
          }else {
              //If your data is utf8 encoded string
              let queryUrl = URL(string: url.absoluteString + "?" + String(decoding: requestBody, as: UTF8.self))
              parameters = queryUrl?.queryParameters
          }
      }
      //Your post payload in parameters
      //Handle it the way you want.
  }
  decisionHandler(.allow)
}


Q: Count occurrences of each unique value within a column pandas I have a dataset which looks as follows:

Station
Month
Year

A
Jan
2021

A
Feb
2021

A
Jan
2021

B
Mar
2021

B
Mar
2021

C
Apr
2021

D
Feb
2021

I am looking to write some code which can identify each unique value within a row and tell me how often it occurs.
eg. For the month column the code should tell be Jan occurs twice, Feb occurs twice, Mar occurs twice and Apr occurs once. It would output similar for the Station and Year columns.
My code looks like this at present, but is not working as intended:
import pandas as pd
for col in df["Month"]:
print((df["Month"])[col].unique())

A: Try value_counts
df["Month"].value_counts()


Q: Internal Server Error when submitting a page I am working a portal website. Here I have forget password page. In this forget password page will have a form with email id. When user gives his email and click on submit it will send an email.
But now actual problem is in my localhost it is working fine. But when I uploaded into the server. And when I open the page forget password and gave email id and clicked on submit then it showing internal server error as shown in the following image.

And the code I have written is
<?php
ini_set('display_errors', 1);
include 'header-employer.php';
echo "In forget password page";

?>

<?php
if(isset($_POST['sub']) && $_POST['sub']=="Submit")
{
    echo "entered into forget password";
    $email=mysql_real_escape_string(trim($_POST['email']));
    echo "email in forget pwd page:".$email;
    $email_fetch=$con->getdata("select * from employers where email='{$email}'");
    $rows=mysql_num_rows($email_fetch);
    print_r($rows);
    if($rows==0)
    {
        echo '';
    }
    else
    {
        function randomcode()
            {
                $chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz0123456789";
                $i = 0;
                $vcode = '' ;
                while($i < 7)
                {
                    $num = mt_rand(0,61);
                    $tmp = substr($chars, $num, 1);
                    $vcode = $vcode . $tmp;
                    $i++;
                }
                return $vcode;
            }
            $msg=randomcode();
            $secure_msg=md5($msg);
            echo "update employers set pass='$secure_msg' where email='{$email}'";
            $update_pass=$con->setdata("update employers set pass='$secure_msg' where email='{$email}'");

                        $user_subject='Your New Password';
                        $user_message='
                        <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<meta name="viewport" content="width=device-width"/>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Job Portal</title>

Dear User,

Your password was reset at www.jobwhizz.com

New password is '.$msg.'

Connect with Us:

Facebook Twitter Google+

Contact Info:

Phone: 408.341.0600

Email: support@jobwhizz.com

                        ';
                        echo "sending email";
                        $mailto_user=mail($email, $user_subject, $user_message, $headers);

                        //var_dump($mailto_user);
                        if($mailto_user)
                        {
                            echo '';
                            //echo "";
                            header('location: employer-signin.php');
                        }
    }
}
?>

Forgot Password ?

<form class="form-horizontal" method="post" action="" onsubmit="return valid_details();" autocomplete="off">

        Your Registered Email Id

      <input type="text" class="form-control" name="email" id="email" placeholder="Registered Email Id">

      <input type="submit" class="btn btn-warning" name="sub" value="Submit">

</form>

<?php include 'footer.php'; ?>

Can anyone help me please...............
Thanks in advance...............!

A: Here are few things you can try.

*

*Try adding error_reporting('E_ALL'); on top of the script.

*Check your web server's configuration  (htaccess, virtualhost etc).

*(more likely cause) Since, you are using mail() function, it could be causing the error. Check your server's mail configuration. More info: PHP's mail() function causes a 500 Internal Server Error only after a certain point in the code

*Compare your server's configuration with your localhost's configuration.


Q: JanusGraph + Cassandra (Generic questions) I have a few questions regarding the integration of the two tools. Not technical questions and how to setup( i will have my fun with that later ) but more on the course of the project and the direction, seeing that JanusGraph is still very young.
I am starting a new project and already decided to use Cassandra for storage and using a graph on top sounds very appealing to me.
A couple of things that i would like to know in advance before i take that road.

*

*JanusGraph is very young and it picks up from where Titan left about a year or so ago. There is gap there but the fact that is part of the Linux Foundation and all the big players are going to support it sounds promising. Is it safe to assume at this point that JanusGraph is here to stay? Would it be safe to depend on Janus as a startup project? And follow development of course and be up to date as much as possible.

*Cassandra. Titan/JanusGraph integrates with Cassandra 2.1.9 using the thrift api which will be deprecated eventually in Cassandra 4. I know that work is being done at the moment to make janus work with Cassandra 3 and eventually work with CQL as well. Is it safe to start with existing janus and Cassandra 2.1.9 and deal with the migration later on? Will it be a huge task for a startup to handle?

*Production ready JanusGraph.(This question relates to any kind of software in it's early stages and whether it's safe for a start up to use). As i understand it, it will take some time for JanusGraph to be production ready and catch up with the rest of the tools it integrates with( although work is being done as we speak:)). Again would it be safe to start using Janus at this point and follow development and finally migrate to a production ready version? What is the overall roadmap for JanusGraph?
My concern in general is whether the combination of the tools is a safe choice for a start up. The whole stack is already new to us and we are excited to try and learn but we will hit a migration period pretty quickly. Is it something that you would do/recommend? Is it a suicide?
Please share your thoughts and keep in mind that it doesn't have to be about the stack i am talking about. It could be any startup company dealing with any kind of software in its early stages.
Cheers

A: Full disclosure, I'm a developer for JanusGraph on Compose.

*

*It's as safe as any other OSS software project with a large amount of backers. Everyone could jump on some new toy tomorrow, but I doubt it. Companies are putting money into it and the development community is very active.

*There is a CQL backend for Janus that's compatible with the Thrift data model. Migration to CQL should be simple and pretty painless when 0.2.0 is released.

*I know there are already people using Titan for production applications. With JanusGraph being forked from Titan, I think it's pretty reasonable to start in with JanusGraph from everything I've seen. As far as a roadmap, I'd check out the JanusGraph mailing list (dev/users) and see what's going on and what's being talked about.

A: Disclosure: I am one of the co-founders of the JanusGraph project; I am also seeking out and adding production users to our GitHub repo and website, so I may be slightly biased. :)
Regarding your questions:

*

*Is it safe to use?
The project is young, but it is built on a foundation of Titan, a very popular graph database that's been around since 2012 and has already been running in production. We have contributors from a number of well-known companies, and several companies are building their business-critical applications directly on JanusGraph, e.g.,

*

*GRAKN.AI is building their knowledge graph on JanusGraph

*IBM's Compose.io has built a managed JanusGraph service

*Uber is already running JanusGraph in production (having previously run Titan)

*several other companies  run JanusGraph as a core part of their production environment

We are also starting to identify companies who will provide consulting services around JanusGraph in case someone needs production-level support for their own self-managed deployments.
So as you can see, there is significant interest in and support for this project.

*Cassandra upgrade
@pantalohnes answered this question; I won't repeat it here.

*Production readiness
As I linked above (GitHub repo and website), we already have production users of JanusGraph which you can find there. Those are just the companies that are publicly willing to lend their name/logo to the project; I'm sure there are more. Also, Titan has been running in many production environments for several years; JanusGraph is a more up-to-date version of Titan, despite the low version number.
I am also speaking with other companies who are planning to migrate to JanusGraph soon; look for announcements via the @JanusGraph Twitter handle to learn about more production deployments.


Q: write! macro does not compile in a separate method when taking reference This is my code:
use std::fs::File;
use std::io::Write;

fn main() {
    let f = File::create("").unwrap();

    // Compiles
    write!(&f, "hi").unwrap();

    write_hi(&f);
}

fn write_hi(f: &File) {
    // Doesn't compile (cannot borrow `f` as mutable, as it is not declared as mutable)
    write!(f, "hi").unwrap();
}

When I have this line without the file being a parameter value, it compiles:
write!(&f, "hi").unwrap();

However, when the f is a parameter value, I get a compile error. It works when I make some mutability changes in the declaration of the f variable and method parameter, but isn't that odd?
Why doesn't the write! macro work on a non-mutable reference when it is used as a parameter value, like it does compile when the referencing variable is declared in the same method?

A: The write! macro internally uses write_fmt which takes a &mut self. Your write!(&f, ...) is actually the odd one since the object being written to should be mutable, however you have an immutable File.
How does this work then? Write has an additional implementation for &Files. So you can write to a mutable reference to an immutable file. I'm not positive, but I believe this was added as a deliberate workaround to allow Files to be written to immutably.
It works in the first case because &f creates a temporary, which can be used mutably. The reason why write!(f, ... (with f being &File) doesn't work is because f is a variable that write_fmt wants to modify, so it needs to be mut.
fn write_hi(mut f: &File) {
         // ^^^
    write!(f, "hi").unwrap();
}

See also:

*

*Why is it possible to implement Read on an immutable reference to File?